A mathematical analysis of the effects of Hebbian learning rules on the dynamics and structure of discrete-time random recurrent neural networks.
We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule, including passive forgetting and different timescales, for neuronal activity and learning dynamics. Previous numerical work has reported that Hebbian learning drives the system from chaos to a steady state through a sequence of bifurcations. Here, we interpret these results mathematically and show that these effects, involving a complex coupling between neuronal dynamics and synaptic graph structure, can be analyzed using Jacobian matrices, which introduce both a structural and a dynamical point of view on neural network evolution. Furthermore, we show that sensitivity to a learned pattern is maximal when the largest Lyapunov exponent is close to 0. We discuss how neural networks may take advantage of this regime of high functional interest.
Siri
Benoît
B
Team Alchemy, INRIA, Parc Club Orsay Université, Orsay Cedex, France. benoit.siri@gmail.com
Berry
Hugues
H
Cessac
Bruno
B
Delord
Bruno
B
Quoy
Mathias
M
D000200:Action Potentials / Q000502:physiology; D000818:Animals; D019277:Entropy; D005246:Feedback; D006801:Humans; D007858:Learning / Q000502:physiology*; D008433:Mathematics; D009415:Nerve Net / Q000502:physiology*; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*; D017711:Nonlinear Dynamics; D013569:Synapses / Q000502:physiology; D013997:Time Factors