title: Dynamical regimes in neural network models of matching behavior.
abstract: The matching law constitutes a quantitative description of choice behavior that is often observed in foraging tasks. According to the matching law, organisms distribute their behavior across available response alternatives in the same proportion that reinforcers are distributed across those alternatives. Recently a few biophysically plausible neural network models have been proposed to explain the matching behavior observed in the experiments. Here we study systematically the learning dynamics of these networks while performing a matching task on the concurrent variable interval (VI) schedule. We found that the model neural network can operate in one of three qualitatively different regimes depending on the parameters that characterize the synaptic dynamics and the reward schedule: (1) a matching behavior regime, in which the probability of choosing an option is roughly proportional to the baiting fractional probability of that option; (2) a perseverative regime, in which the network tends to make always the same decision; and (3) a tristable regime, in which the network can either perseverate or choose the two targets randomly approximately with the same probability. Different parameters of the synaptic dynamics lead to different types of deviations from the matching law, some of which have been observed experimentally. We show that the performance of the network depends on the number of stable states of each synapse and that bistable synapses perform close to optimal when the proper learning rate is chosen. Because our model provides a link between synaptic dynamics and qualitatively different behaviors, this work provides us with insight into the effects of neuromodulators on adaptive behaviors and psychiatric disorders.
authors.affiliation: Center for Theoretical Neuroscience, Department of Neuroscience, Columbia University Medical Center, New York, NY 10032, and Department of Physics, Columbia University, New York, NY 10027, U.S.A. ki2151@columbia.edu.
authors.forename: Kiyohito
authors.initials: K
authors.lastname: Iigaya
authors.forename: Stefano
authors.initials: S
authors.lastname: Fusi
mesh_terms: D000818:Animals; D001921:Brain / Q000502:physiology*; D002755:Choice Behavior / Q000502:physiology*; D003657:Decision Making / Q000502:physiology; D006801:Humans; D008959:Models, Neurological; D016571:Neural Networks, Computer
references.citation: Neuron. 2007 Apr 19;54(2):319-33
references.pmid: 17442251
references.citation: Neuron. 2008 May 8;58(3):451-63
references.pmid: 18466754
references.citation: PLoS Comput Biol. 2008 Mar 07;4(3):e1000007
references.pmid: 18369414
references.citation: J Neurosci. 2009 Dec 2;29(48):15104-14
references.pmid: 19955362
references.citation: J Neurosci. 2013 Jan 23;33(4):1521-34
references.pmid: 23345226
references.citation: J Neurosci. 2010 Sep 15;30(37):12366-78
references.pmid: 20844132
references.citation: Proc Natl Acad Sci U S A. 2006 Oct 10;103(41):15224-9
references.pmid: 17008410
references.citation: Nat Neurosci. 2007 Apr;10(4):485-93
references.pmid: 17351638
references.citation: Neural Netw. 2006 Oct;19(8):1075-90
references.pmid: 17015181
references.citation: Neuron. 2005 Feb 17;45(4):599-611
references.pmid: 15721245
references.citation: J Exp Anal Behav. 2005 Nov;84(3):581-617
references.pmid: 16596981
references.citation: Science. 2004 Jun 18;304(5678):1782-7
references.pmid: 15205529
references.citation: Brain Res. 2009 Nov 24;1299:95-117
references.pmid: 19646968
references.citation: J Exp Anal Behav. 1961 Jul;4:267-72
references.pmid: 13713775
references.citation: PLoS One. 2008;3(11):e3795
references.pmid: 19030101
references.citation: J Exp Psychol Anim Behav Process. 2001 Oct;27(4):354-72
references.pmid: 11676086
references.citation: Neural Comput. 2012 May;24(5):1230-70
references.pmid: 22295982
references.citation: Brain Res Cogn Brain Res. 2004 Dec;22(1):45-58
references.pmid: 15561500
references.citation: Front Comput Neurosci. 2010 Jun 17;4:17
references.pmid: 20617184
references.citation: Proc Natl Acad Sci U S A. 2001 Mar 27;98(7):4265-70
references.pmid: 11259678
references.citation: Nat Neurosci. 2012 Jun 03;15(7):1040-6
references.pmid: 22660479
references.citation: J Neurosci. 2006 Apr 5;26(14):3731-44
references.pmid: 16597727
references.citation: Nat Neurosci. 2007 Sep;10(9):1214-21
references.pmid: 17676057
references.citation: Cognition. 1994 Apr-Jun;50(1-3):151-70
references.pmid: 8039358
references.citation: Neural Comput. 2009 Oct;21(10):2755-73
references.pmid: 19635021
references.citation: Front Comput Neurosci. 2013 Apr 12;7:32
references.pmid: 23641210
references.citation: PLoS Comput Biol. 2008 Dec;4(12):e1000248
references.pmid: 19112486
references.citation: Neural Comput. 2008 Jan;20(1):227-51
references.pmid: 18045007
references.citation: Neuron. 2002 Dec 5;36(5):955-68
references.pmid: 12467598
references.citation: PLoS Comput Biol. 2013;9(7):e1003146
references.pmid: 23935470
references.citation: J Exp Anal Behav. 2005 Nov;84(3):555-79
references.pmid: 16596980