Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning.
Individual behavioral performance during learning is known to be affected by modulatory factors, such as stress and motivation, and by genetic predispositions that influence sensitivity to these factors. Despite numerous studies, no integrative framework is available that could predict how a given animal would perform a certain learning task in a realistic situation. We found that a simple reinforcement learning model can predict mouse behavior in a hole-box conditioning task if model metaparameters are dynamically controlled on the basis of the mouse's genotype and phenotype, stress conditions, recent performance feedback and pharmacological manipulations of adrenergic alpha-2 receptors. We find that stress and motivation affect behavioral performance by altering the exploration-exploitation balance in a genotype-dependent manner. Our results also provide computational insights into how an inverted U-shape relation between stress/arousal/norepinephrine levels and behavioral performance could be explained through changes in task performance accuracy and future reward discounting.
Luksys
Gediminas
G
Laboratory of Computational Neuroscience, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland. eurogedi@hotmail.com
Gerstner
Wulfram
W
Sandi
Carmen
C
D000818:Animals; D001522:Behavior, Animal / Q000502:physiology*; D003198:Computer Simulation; D003214:Conditioning, Classical / Q000502:physiology; D005106:Exploratory Behavior / Q000502:physiology; D030141:Feedback, Psychological; D005838:Genotype; D007858:Learning / Q000502:physiology*; D008297:Male; D051379:Mice; D008810:Mice, Inbred C57BL; D008811:Mice, Inbred DBA; D008960:Models, Psychological; D009042:Motivation; D009638:Norepinephrine / Q000378:metabolism*; D010551:Personality / Q000235:genetics; D010641:Phenotype; D018341:Receptors, Adrenergic, alpha-2 / Q000378:metabolism; D012054:Reinforcement, Psychology; D013045:Species Specificity; D013315:Stress, Psychological / Q000503:physiopathology*
D018341:Receptors, Adrenergic, alpha-2; D009638:Norepinephrine
Nat Rev Neurosci. 2002 Jun;3(6):453-62
12042880
Science. 2005 Nov 25;310(5752):1337-40
16311337
Neural Netw. 2002 Jun-Jul;15(4-6):495-506
12371507
Prog Neurobiol. 2007 Oct;83(2):69-91
17681661
Pharmacol Biochem Behav. 2008 Feb;88(4):403-6
17945338
Life Sci I. 1972 Jun 15;11(12):565-72
4144315
Nat Neurosci. 2007 Sep;10(9):1214-21
17676057
Trends Pharmacol Sci. 2008 Apr;29(4):192-9
18304658
Behav Pharmacol. 2003 Feb;14(1):41-8
12576880
Nat Neurosci. 2005 Mar;8(3):365-71
15696163
Eur J Neurosci. 1997 Apr;9(4):637-42
9153570
Trends Cogn Sci. 2006 Apr;10(4):152-8
16513410
Neural Plast. 2007;2007:78970
18060012
Nat Rev Neurosci. 2009 Mar;10(3):211-23
19190638
Genes Brain Behav. 2002 Jan;1(1):55-69
12886950
Psychopharmacology (Berl). 2003 Apr;166(4):373-82
12601501
Science. 2004 Dec 10;306(5703):1944-7
15591205
Nat Neurosci. 2004 Aug;7(8):887-93
15235607
Nat Rev Neurosci. 2000 Oct;1(1):11-20
11252764
Neurobiol Learn Mem. 2006 Sep;86(2):150-9
16580234
Proc Natl Acad Sci U S A. 2007 Oct 9;104(41):16311-6
17913879
Trends Neurosci. 1998 Dec;21(12):505-9
9881846
Neuron. 2005 May 19;46(4):681-92
15944135
J Neurosci. 2000 Sep 15;20(18):6983-8
10995843
Neuroscience. 1996 Jan;70(1):1-5
8848115
Science. 2007 Mar 2;315(5816):1267-70
17332411
Nat Neurosci. 2008 Apr;11(4):410-6
18368048
Neural Plast. 2007;2007:35457
17502910
Prog Brain Res. 2000;126:165-82
11105646
Annu Rev Neurosci. 2005;28:403-50
16022602
Psychol Rev. 2001 Jul;108(3):624-52
11488380
J Neurosci. 2008 Apr 23;28(17):4528-32
18434531
Behav Brain Res. 2002 Mar 10;130(1-2):103-9
11864725
Brain Res. 2007 Sep 12;1169:69-76
17706947
Neuron. 2002 Oct 10;36(2):285-98
12383782
Science. 1997 Mar 14;275(5306):1593-9
9054347