Hypothesis of Neuron Activation According to the Laws of Symmetry
The paper discusses the main activation functions in modern neural networks and their disadvantages. It is concluded that all of them have one drawback, which is the inability to interpret the received signals, these are just normalized values of the weighted sum of synapses. A table of symmetries (automorphisms) and their role in semiotic analysis and linguistics are considered. Linguistics contains universals, which, even in the superficial analysis, are symmetries. Therefore, semiotic analysis is a mathematical method, just as linguistics is an exact science, subject to the laws of set theory and universal algebra. An assumption is made about the possibility of using pragmatic analysis and the mechanism of symmetries in neural networks. A new approach is proposed, which includes the grouping of neurons in the hidden layer by the form of symmetry (automorphism) and the use of three-phase activation functions for each group, which characterize the manifestation of automorphism properties of this group. Each group of neurons has its own memory for storing frequent signals, which further generate symbol chains. At the initial stage, two groups of symmetries are taken - reversible and mirror. The proposed approach can make neural networks more accessible for understanding because of the interpretability of signals.