Individual differences and the neural representations of reward expectation and reward prediction error.
Reward expectation and reward prediction errors are thought to be critical for dynamic adjustments in decision-making and reward-seeking behavior, but little is known about their representation in the brain during uncertainty and risk-taking. Furthermore, little is known about what role individual differences might play in such reinforcement processes. In this study, it is shown behavioral and neural responses during a decision-making task can be characterized by a computational reinforcement learning model and that individual differences in learning parameters in the model are critical for elucidating these processes. In the fMRI experiment, subjects chose between high- and low-risk rewards. A computational reinforcement learning model computed expected values and prediction errors that each subject might experience on each trial. These outputs predicted subjects' trial-to-trial choice strategies and neural activity in several limbic and prefrontal regions during the task. Individual differences in estimated reinforcement learning parameters proved critical for characterizing these processes, because models that incorporated individual learning parameters explained significantly more variance in the fMRI data than did a model using fixed learning parameters. These findings suggest that the brain engages a reinforcement learning process during risk-taking and that individual differences play a crucial role in modeling this process.
Cohen
Michael X
MX
Department of Epilepsy, University of Bonn, Sigmund-Freud-Strasse 25, Bonn, Germany. mcohen@ucdavis.edu
D000328:Adult; D001290:Attitude; D001921:Brain / Q000502:physiology*; D003657:Decision Making; D005260:Female; D006801:Humans; D007858:Learning; D008279:Magnetic Resonance Imaging; D008297:Male; D017397:Prefrontal Cortex / Q000502:physiology; D011446:Prospective Studies; D012054:Reinforcement, Psychology; D012201:Reward; D055815:Young Adult
Neuron. 2005 Jul 21;47(2):295-306
16039570
Cereb Cortex. 2000 Mar;10(3):263-71
10731221
Neural Comput. 2005 Feb;17(2):245-319
15720770
Science. 1997 Feb 28;275(5304):1293-5
9036851
Cereb Cortex. 2000 Mar;10(3):295-307
10731224
Nature. 2000 Mar 23;404(6776):390-4
10746726
Prog Brain Res. 2000;126:193-215
11105648
J Pers. 2000 Dec;68(6):999-1029
11130742
Drug Alcohol Depend. 2001 Jun 1;63(1):29-38
11297829
Nature. 2001 Jul 5;412(6842):43-8
11452299
Nat Rev Neurosci. 2002 Mar;3(3):243-9
11994756
Nat Rev Neurosci. 2002 Jul;3(7):563-73
12094212
Neural Netw. 2002 Jun-Jul;15(4-6):535-47
12371510
Neural Netw. 2002 Jun-Jul;15(4-6):603-16
12371515
Psychol Rev. 2002 Oct;109(4):679-709
12374324
Neuron. 2002 Oct 10;36(2):265-84
12383781
Neurobiol Learn Mem. 2003 Jan;79(1):118-21
12482686
Am J Med Genet B Neuropsychiatr Genet. 2003 Jan 1;116B(1):103-25
12497624
Neuron. 2003 Apr 24;38(2):329-37
12718865
Neuron. 2003 Apr 24;38(2):339-46
12718866
Ann N Y Acad Sci. 2003 Apr;985:356-69
12724171
Cereb Cortex. 2003 Oct;13(10):1064-71
12967923
Nat Neurosci. 2004 Apr;7(4):404-10
15004564
Neuroimage. 2004 Apr;21(4):1639-51
15050587
Curr Opin Neurobiol. 2004 Apr;14(2):139-47
15082317
Science. 2004 Apr 16;304(5669):452-4
15087550
Nature. 2004 Jun 10;429(6992):664-7
15190354
Science. 2004 Jun 18;304(5678):1782-7
15205529
Am J Drug Alcohol Abuse. 2004 May;30(2):409-28
15230083
Nat Neurosci. 2004 Aug;7(8):887-93
15235607
Nature. 2004 Oct 14;431(7010):760-7
15483596
Int J Addict. 1979 Jul;14(5):607-26
383632
Neuroscience. 1991;42(1):1-18
1830641
Rev Neurosci. 1995 Jan-Mar;6(1):63-85
7633641
J Neurosci. 1996 Mar 1;16(5):1936-47
8774460
Science. 1997 Mar 14;275(5306):1593-9
9054347
Alcohol. 1998 Jul;16(1):33-45
9650634
J Neurophysiol. 1998 Jul;80(1):1-27
9658025
Neuroimage. 1998 Nov;8(4):360-9
9811554
Nature. 1999 Apr 22;398(6729):704-8
10227292
Neuron. 2005 Jul 7;47(1):129-41
15996553
Brain Res Cogn Brain Res. 2005 Oct;25(2):416-30
16095886
Science. 2005 Nov 25;310(5752):1337-40
16311337
Neuron. 2006 Jan 5;49(1):157-66
16387647
J Neurophysiol. 2006 Feb;95(2):948-59
16192338
Hum Brain Mapp. 2006 Apr;27(4):306-13
16092133
Neuroimage. 2006 Apr 1;30(2):668-77
16321546
Neuroimage. 2006 Jun;31(2):790-5
16487726
J Neurosci. 2007 Jan 10;27(2):371-8
17215398
Hum Brain Mapp. 2007 Apr;28(4):294-302
16779798
Cogn Affect Behav Neurosci. 2005 Jun;5(2):117-26
16180619