Large memory capacity in chaotic artificial neural networks: a view of the anti-integrable limit.
In the literature, it was reported that the chaotic artificial neural network model with sinusoidal activation functions possesses a large memory capacity as well as a remarkable ability of retrieving the stored patterns, better than the conventional chaotic model with only monotonic activation functions such as sigmoidal functions. This paper, from the viewpoint of the anti-integrable limit, elucidates the mechanism inducing the superiority of the model with periodic activation functions that includes sinusoidal functions. Particularly, by virtue of the anti-integrable limit technique, this paper shows that any finite-dimensional neural network model with periodic activation functions and properly selected parameters has much more abundant chaotic dynamics that truly determine the model's memory capacity and pattern-retrieval ability. To some extent, this paper mathematically and numerically demonstrates that an appropriate choice of the activation functions and control scheme can lead to a large memory capacity and better pattern-retrieval ability of the artificial neural network models.
Lin
Wei
W
Key Laboratory of Mathematics for Nonlinear Sciences (Fudan University), Chinese Ministry of Education, China, School of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University, Shanghai 200433, China. wlin@fudan.edu.cn
Chen
Guanrong
G
D000200:Action Potentials; D000465:Algorithms; D003198:Computer Simulation; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D010507:Periodicity; D013997:Time Factors