Changes in corticostriatal connectivity during reinforcement learning in humans.
Many computational models assume that reinforcement learning relies on changes in synaptic efficacy between cortical regions representing stimuli and striatal regions involved in response selection, but this assumption has thus far lacked empirical support in humans. We recorded hemodynamic signals with fMRI while participants navigated a virtual maze to find hidden rewards. We fitted a reinforcement-learning algorithm to participants' choice behavior and evaluated the neural activity and the changes in functional connectivity related to trial-by-trial learning variables. Activity in the posterior putamen during choice periods increased progressively during learning. Furthermore, the functional connections between the sensorimotor cortex and the posterior putamen strengthened progressively as participants learned the task. These changes in corticostriatal connectivity differentiated participants who learned the task from those who did not. These findings provide a direct link between changes in corticostriatal connectivity and learning, thereby supporting a central assumption common to several computational models of reinforcement learning.
Department of Psychiatry, New York State Psychiatric Institute and College of Physicians and Surgeons, Columbia University, New York, New York.
Guillermo
G
Horga
Tiago V
TV
Maia
Rachel
R
Marsh
Xuejun
X
Hao
Dongrong
D
Xu
Yunsuo
Y
Duan
Gregory Z
GZ
Tau
Barbara
B
Graniello
Zhishun
Z
Wang
Alayar
A
Kangarlu
Diana
D
Martinez
Mark G
MG
Packard
Bradley S
BS
Peterson
D000328:Adult; D000465:Algorithms; D001931:Brain Mapping; D002560:Cerebrovascular Circulation / Q000502:physiology; D002755:Choice Behavior / Q000502:physiology; D005260:Female; D006439:Hemodynamics; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D018782:Maze Learning / Q000502:physiology*; D008959:Models, Neurological; D009434:Neural Pathways / Q000098:blood supply / Q000502:physiology; D009483:Neuropsychological Tests; D011601:Psychophysics; D011699:Putamen / Q000098:blood supply / Q000502:physiology*; D012054:Reinforcement, Psychology; D066191:Sensorimotor Cortex / Q000098:blood supply / Q000502:physiology*; D014584:User-Computer Interface
computational model-based fMRI; functional connectivity; putamen; reinforcement learning