title: On the Maximum Storage Capacity of the Hopfield Model.
abstract: Recurrent neural networks (RNN) have traditionally been of great interest for their capacity to store memories. In past years, several works have been devoted to determine the maximum storage capacity of RNN, especially for the case of the Hopfield network, the most popular kind of RNN. Analyzing the thermodynamic limit of the statistical properties of the Hamiltonian corresponding to the Hopfield neural network, it has been shown in the literature that the retrieval errors diverge when the number of stored memory patterns (P) exceeds a fraction (≈ 14%) of the network sizeN. In this paper, we study the storage performance of a generalized Hopfield model, where the diagonal elements of the connection matrix are allowed to be different from zero. We investigate this model at finiteN. We give an analytical expression for the number of retrieval errors and show that, by increasing the number of stored patterns over a certain threshold, the errors start to decrease and reach values below unit forP≫N. We demonstrate that the strongest trade-off between efficiency and effectiveness relies on the number of patterns (P) that are stored in the network by appropriately fixing the connection weights. WhenP≫Nand the diagonal elements of the adjacency matrix are not forced to be zero, the optimal storage capacity is obtained with a number of stored memories much larger than previously reported. This theory paves the way to the design of RNN with high storage capacity and able to retrieve the desired pattern without distortions.
authors.affiliation: Center for Life Nanoscience, Istituto Italiano di Tecnologia Rome, Italy.
authors.forename: Viola
authors.initials: V
authors.lastname: Folli
authors.affiliation: Center for Life Nanoscience, Istituto Italiano di Tecnologia Rome, Italy.
authors.forename: Marco
authors.initials: M
authors.lastname: Leonetti
authors.affiliation: Center for Life Nanoscience, Istituto Italiano di TecnologiaRome, Italy; Department of Physics, Sapienza University of RomeRome, Italy.
authors.forename: Giancarlo
authors.initials: G
authors.lastname: Ruocco
keywords: Hopfield model; feed-forward structure; maximum storage memory; random recurrent network; retrieval error