title: Memory capacity of networks with stochastic binary synapses.
abstract: In standard attractor neural network models, specific patterns of activity are stored in the synaptic matrix, so that they become fixed point attractors of the network dynamics. The storage capacity of such networks has been quantified in two ways: the maximal number of patterns that can be stored, and the stored information measured in bits per synapse. In this paper, we compute both quantities in fully connected networks of N binary neurons with binary synapses, storing patterns with coding level [Formula: see text], in the large [Formula: see text] and sparse coding limits ([Formula: see text]). We also derive finite-size corrections that accurately reproduce the results of simulations in networks of tens of thousands of neurons. These methods are applied to three different scenarios: (1) the classic Willshaw model, (2) networks with stochastic learning in which patterns are shown only once (one shot learning), (3) networks with stochastic learning in which patterns are shown multiple times. The storage capacities are optimized over network parameters, which allows us to compare the performance of the different models. We show that finite-size effects strongly reduce the capacity, even for networks of realistic sizes. We discuss the implications of these results for memory storage in the hippocampus and cerebral cortex.
authors.affiliation: UMR 8118, CNRS, Université Paris Descartes, Paris, France; Departments of Statistics and Neurobiology, University of Chicago, Chicago, Illinois, United States of America.
authors.forename: Alexis M
authors.initials: AM
authors.lastname: Dubreuil
authors.affiliation: Departments of Statistics and Computer Science, University of Chicago, Chicago, Illinois, United States of America.
authors.forename: Yali
authors.initials: Y
authors.lastname: Amit
authors.affiliation: UMR 8118, CNRS, Université Paris Descartes, Paris, France; Departments of Statistics and Neurobiology, University of Chicago, Chicago, Illinois, United States of America.
authors.forename: Nicolas
authors.initials: N
authors.lastname: Brunel
mesh_terms: D000818:Animals; D002540:Cerebral Cortex / Q000502:physiology; D019295:Computational Biology; D006624:Hippocampus / Q000502:physiology; D008568:Memory / Q000502:physiology*; D008959:Models, Neurological; D009415:Nerve Net / Q000502:physiology*; D009474:Neurons / Q000502:physiology; D051381:Rats; D013569:Synapses / Q000502:physiology*
references.citation: Nat Neurosci. 2004 Mar;7(3):286-94
references.pmid: 14983183
references.citation: PLoS Comput Biol. 2007 Sep;3(9):1679-700
references.pmid: 17845070
references.citation: Phys Rev A Gen Phys. 1986 Sep;34(3):2571-2574
references.pmid: 9897569
references.citation: Nature. 1988 Oct 27;335(6193):817-20
references.pmid: 3185711
references.citation: Neural Comput. 1998 Aug 15;10(6):1321-71
references.pmid: 9698348
references.citation: Neuron. 2009 Apr 30;62(2):242-53
references.pmid: 19409269
references.citation: PLoS Comput Biol. 2008 Nov;4(11):e1000230
references.pmid: 19043540
references.citation: J Comput Neurosci. 2011 Jun;30(3):699-720
references.pmid: 20978831
references.citation: Trends Neurosci. 2004 Dec;27(12):744-50
references.pmid: 15541515
references.citation: Neural Comput. 2010 Mar;22(3):660-88
references.pmid: 19842984
references.citation: Proc Natl Acad Sci U S A. 1994 Jul 19;91(15):7041-5
references.pmid: 8041742
references.citation: PLoS Comput Biol. 2013;9(7):e1003146
references.pmid: 23935470
references.citation: Nature. 1988 Jan 7;331(6151):68-70
references.pmid: 3340148
references.citation: Cereb Cortex. 1997 Apr-May;7(3):237-52
references.pmid: 9143444
references.citation: Cereb Cortex. 2008 Jan;18(1):67-77
references.pmid: 17490993
references.citation: Neural Comput. 2010 Feb;22(2):289-341
references.pmid: 19925281
references.citation: Science. 1996 Dec 6;274(5293):1724-6
references.pmid: 8939866
references.citation: Proc Natl Acad Sci U S A. 1998 Apr 14;95(8):4732-7
references.pmid: 9539807
references.citation: J Neurophysiol. 1995 Jul;74(1):162-78
references.pmid: 7472321
references.citation: Science. 1971 Aug 13;173(3997):652-4
references.pmid: 4998337
references.citation: Proc Natl Acad Sci U S A. 2007 Jun 26;104(26):11079-84
references.pmid: 17581884
references.citation: Proc Natl Acad Sci U S A. 1982 Apr;79(8):2554-8
references.pmid: 6953413
references.citation: Neuron. 1995 Mar;14(3):477-85
references.pmid: 7695894
references.citation: Proc Natl Acad Sci U S A. 2005 Jul 5;102(27):9679-84
references.pmid: 15983385
references.citation: Nature. 1969 Jun 7;222(5197):960-2
references.pmid: 5789326
references.citation: J Neurosci. 2011 Jun 29;31(26):9481-8
references.pmid: 21715613
references.citation: Network. 1998 Feb;9(1):123-52
references.pmid: 9861982
references.citation: Annu Rev Neurosci. 1993;16:245-63
references.pmid: 8460893
references.citation: J Math Biol. 1977 Oct 20;4(4):303-21
references.pmid: 925522
references.citation: J Comput Neurosci. 2000 May-Jun;8(3):183-208
references.pmid: 10809012