title: Noise tolerance of attractor and feedforward memory models.
abstract: In short-term memory networks, transient stimuli are represented by patterns of neural activity that persist long after stimulus offset. Here, we compare the performance of two prominent classes of memory networks, feedback-based attractor networks and feedforward networks, in conveying information about the amplitude of a briefly presented stimulus in the presence of gaussian noise. Using Fisher information as a metric of memory performance, we find that the optimal form of network architecture depends strongly on assumptions about the forms of nonlinearities in the network. For purely linear networks, we find that feedforward networks outperform attractor networks because noise is continually removed from feedforward networks when signals exit the network; as a result, feedforward networks can amplify signals they receive faster than noise accumulates over time. By contrast, attractor networks must operate in a signal-attenuating regime to avoid the buildup of noise. However, if the amplification of signals is limited by a finite dynamic range of neuronal responses or if noise is reset at the time of signal arrival, as suggested by recent experiments, we find that attractor networks can outperform feedforward ones. Under a simple model in which neurons have a finite dynamic range, we find that the optimal attractor networks are forgetful if there is no mechanism for noise reduction with signal arrival but nonforgetful (perfect integrators) in the presence of a strong reset mechanism. Furthermore, we find that the maximal Fisher information for the feedforward and attractor networks exhibits power law decay as a function of time and scales linearly with the number of neurons. These results highlight prominent factors that lead to trade-offs in the memory performance of networks with different architectures and constraints, and suggest conditions under which attractor or feedforward networks may be best suited to storing information about previous stimuli.
authors.affiliation: Center for Neuroscience, University of California, Davis, Davis, CA 95618, USA. sblim@ucdavis.edu
authors.forename: Sukbin
authors.initials: S
authors.lastname: Lim
authors.forename: Mark S
authors.initials: MS
authors.lastname: Goldman
mesh_terms: D025461:Feedback, Physiological / Q000502:physiology*; D008570:Memory, Short-Term / Q000502:physiology*; D008959:Models, Neurological; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*
references.citation: Phys Rev E Stat Nonlin Soft Matter Phys. 2010 Jul;82(1 Pt 1):011903
references.pmid: 20866644
references.citation: Trends Neurosci. 2001 Aug;24(8):455-63
references.pmid: 11476885
references.citation: Cereb Cortex. 2003 Nov;13(11):1185-95
references.pmid: 14576210
references.citation: Vision Res. 1994 Jan;34(1):79-92
references.pmid: 8116271
references.citation: Neuron. 2009 Feb 26;61(4):635-48
references.pmid: 19249282
references.citation: Phys Rev E Stat Nonlin Soft Matter Phys. 2001 Nov;64(5 Pt 1):051904
references.pmid: 11735965
references.citation: Trends Neurosci. 2001 May;24(5):289-94
references.pmid: 11311382
references.citation: Science. 2008 Sep 5;321(5894):1322-7
references.pmid: 18772431
references.citation: Proc Biol Sci. 1997 Dec 22;264(1389):1775-83
references.pmid: 9447735
references.citation: Cereb Cortex. 1997 Apr-May;7(3):237-52
references.pmid: 9143444
references.citation: Neuron. 2003 May 8;38(3):473-85
references.pmid: 12741993
references.citation: Proc Natl Acad Sci U S A. 2008 Dec 2;105(48):18970-5
references.pmid: 19020074
references.citation: Proc Natl Acad Sci U S A. 1996 Nov 26;93(24):13481-6
references.pmid: 8942960
references.citation: Proc Natl Acad Sci U S A. 1996 Nov 12;93(23):13339-44
references.pmid: 8917592
references.citation: J Neurophysiol. 2000 May;83(5):3031-41
references.pmid: 10805698
references.citation: Science. 2008 Jul 4;321(5885):48-50
references.pmid: 18599763
references.citation: Nature. 1994 Jul 14;370(6485):140-3
references.pmid: 8022482
references.citation: J Comput Neurosci. 1998 Dec;5(4):383-405
references.pmid: 9877021
references.citation: Neuron. 2009 Feb 26;61(4):621-34
references.pmid: 19249281
references.citation: Nature. 2010 Nov 18;468(7322):394-9
references.pmid: 20972420
references.citation: Nat Rev Neurosci. 2006 May;7(5):358-66
references.pmid: 16760916
references.citation: Nat Neurosci. 2002 Aug;5(8):775-82
references.pmid: 12134153
references.citation: Cereb Cortex. 2003 Nov;13(11):1162-72
references.pmid: 14576208
references.citation: Nat Neurosci. 2010 Mar;13(3):369-78
references.pmid: 20173745
references.citation: J Comput Neurosci. 1994 Jun;1(1-2):89-107
references.pmid: 8792227
references.citation: J Neurosci. 2010 Jan 6;30(1):350-60
references.pmid: 20053916
references.citation: J Neurophysiol. 2000 Aug;84(2):1035-49
references.pmid: 10938326
references.citation: Curr Opin Neurobiol. 2004 Dec;14(6):675-84
references.pmid: 15582368
references.citation: Science. 2008 Mar 14;319(5869):1543-6
references.pmid: 18339943
references.citation: Neuron. 2008 Apr 10;58(1):15-25
references.pmid: 18400159
references.citation: Annu Rev Neurosci. 2004;27:307-40
references.pmid: 15217335
references.citation: Neuron. 1995 Mar;14(3):477-85
references.pmid: 7695894
references.citation: Vision Res. 1972 Mar;12(3):467-75
references.pmid: 5021911
references.citation: Annu Rev Neurosci. 1989;12:33-45
references.pmid: 2648952
references.citation: PLoS Biol. 2006 Apr;4(4):e92
references.pmid: 16529529
references.citation: Neuron. 2009 Feb 26;61(4):499-501
references.pmid: 19249270
references.citation: Curr Opin Neurobiol. 2003 Apr;13(2):204-11
references.pmid: 12744975
references.citation: Eur J Neurosci. 1998 Feb;10(2):794-7
references.pmid: 9749746
references.citation: Nat Neurosci. 2000 Nov;3 Suppl:1184-91
references.pmid: 11127836
references.citation: Neural Comput. 2012 Oct;24(10):2678-99
references.pmid: 22594828
references.citation: Neural Comput. 1999 Jan 1;11(1):91-101
references.pmid: 9950724
references.citation: Neuron. 2011 Aug 25;71(4):737-49
references.pmid: 21867888
references.citation: J Neurosci. 2006 Apr 5;26(14):3667-78
references.pmid: 16597721
references.citation: Nat Neurosci. 2006 Apr;9(4):534-42
references.pmid: 16547512
references.citation: J Physiol Paris. 2003 Jul-Nov;97(4-6):683-94
references.pmid: 15242674
references.citation: J Neurosci. 1996 Feb 15;16(4):1486-510
references.pmid: 8778300
references.citation: Phys Rev Lett. 2004 Apr 9;92(14):148102
references.pmid: 15089576
references.citation: Nature. 1999 Jun 3;399(6735):470-3
references.pmid: 10365959