title: Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning.
abstract: Individual behavioral performance during learning is known to be affected by modulatory factors, such as stress and motivation, and by genetic predispositions that influence sensitivity to these factors. Despite numerous studies, no integrative framework is available that could predict how a given animal would perform a certain learning task in a realistic situation. We found that a simple reinforcement learning model can predict mouse behavior in a hole-box conditioning task if model metaparameters are dynamically controlled on the basis of the mouse's genotype and phenotype, stress conditions, recent performance feedback and pharmacological manipulations of adrenergic alpha-2 receptors. We find that stress and motivation affect behavioral performance by altering the exploration-exploitation balance in a genotype-dependent manner. Our results also provide computational insights into how an inverted U-shape relation between stress/arousal/norepinephrine levels and behavioral performance could be explained through changes in task performance accuracy and future reward discounting.
authors.affiliation: Laboratory of Computational Neuroscience, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland. eurogedi@hotmail.com
authors.forename: Gediminas
authors.initials: G
authors.lastname: Luksys
authors.forename: Wulfram
authors.initials: W
authors.lastname: Gerstner
authors.forename: Carmen
authors.initials: C
authors.lastname: Sandi
mesh_terms: D000818:Animals; D001522:Behavior, Animal / Q000502:physiology*; D003198:Computer Simulation; D003214:Conditioning, Classical / Q000502:physiology; D005106:Exploratory Behavior / Q000502:physiology; D030141:Feedback, Psychological; D005838:Genotype; D007858:Learning / Q000502:physiology*; D008297:Male; D051379:Mice; D008810:Mice, Inbred C57BL; D008811:Mice, Inbred DBA; D008960:Models, Psychological; D009042:Motivation; D009638:Norepinephrine / Q000378:metabolism*; D010551:Personality / Q000235:genetics; D010641:Phenotype; D018341:Receptors, Adrenergic, alpha-2 / Q000378:metabolism; D012054:Reinforcement, Psychology; D013045:Species Specificity; D013315:Stress, Psychological / Q000503:physiopathology*
chemical_list: D018341:Receptors, Adrenergic, alpha-2; D009638:Norepinephrine
references.citation: Nat Rev Neurosci. 2002 Jun;3(6):453-62
references.pmid: 12042880
references.citation: Science. 2005 Nov 25;310(5752):1337-40
references.pmid: 16311337
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):495-506
references.pmid: 12371507
references.citation: Prog Neurobiol. 2007 Oct;83(2):69-91
references.pmid: 17681661
references.citation: Pharmacol Biochem Behav. 2008 Feb;88(4):403-6
references.pmid: 17945338
references.citation: Life Sci I. 1972 Jun 15;11(12):565-72
references.pmid: 4144315
references.citation: Nat Neurosci. 2007 Sep;10(9):1214-21
references.pmid: 17676057
references.citation: Trends Pharmacol Sci. 2008 Apr;29(4):192-9
references.pmid: 18304658
references.citation: Behav Pharmacol. 2003 Feb;14(1):41-8
references.pmid: 12576880
references.citation: Nat Neurosci. 2005 Mar;8(3):365-71
references.pmid: 15696163
references.citation: Eur J Neurosci. 1997 Apr;9(4):637-42
references.pmid: 9153570
references.citation: Trends Cogn Sci. 2006 Apr;10(4):152-8
references.pmid: 16513410
references.citation: Neural Plast. 2007;2007:78970
references.pmid: 18060012
references.citation: Nat Rev Neurosci. 2009 Mar;10(3):211-23
references.pmid: 19190638
references.citation: Genes Brain Behav. 2002 Jan;1(1):55-69
references.pmid: 12886950
references.citation: Psychopharmacology (Berl). 2003 Apr;166(4):373-82
references.pmid: 12601501
references.citation: Science. 2004 Dec 10;306(5703):1944-7
references.pmid: 15591205
references.citation: Nat Neurosci. 2004 Aug;7(8):887-93
references.pmid: 15235607
references.citation: Nat Rev Neurosci. 2000 Oct;1(1):11-20
references.pmid: 11252764
references.citation: Neurobiol Learn Mem. 2006 Sep;86(2):150-9
references.pmid: 16580234
references.citation: Proc Natl Acad Sci U S A. 2007 Oct 9;104(41):16311-6
references.pmid: 17913879
references.citation: Trends Neurosci. 1998 Dec;21(12):505-9
references.pmid: 9881846
references.citation: Neuron. 2005 May 19;46(4):681-92
references.pmid: 15944135
references.citation: J Neurosci. 2000 Sep 15;20(18):6983-8
references.pmid: 10995843
references.citation: Neuroscience. 1996 Jan;70(1):1-5
references.pmid: 8848115
references.citation: Science. 2007 Mar 2;315(5816):1267-70
references.pmid: 17332411
references.citation: Nat Neurosci. 2008 Apr;11(4):410-6
references.pmid: 18368048
references.citation: Neural Plast. 2007;2007:35457
references.pmid: 17502910
references.citation: Prog Brain Res. 2000;126:165-82
references.pmid: 11105646
references.citation: Annu Rev Neurosci. 2005;28:403-50
references.pmid: 16022602
references.citation: Psychol Rev. 2001 Jul;108(3):624-52
references.pmid: 11488380
references.citation: J Neurosci. 2008 Apr 23;28(17):4528-32
references.pmid: 18434531
references.citation: Behav Brain Res. 2002 Mar 10;130(1-2):103-9
references.pmid: 11864725
references.citation: Brain Res. 2007 Sep 12;1169:69-76
references.pmid: 17706947
references.citation: Neuron. 2002 Oct 10;36(2):285-98
references.pmid: 12383782
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347