Working Memory Load Strengthens Reward Prediction Errors.
Reinforcement learning (RL) in simple instrumental tasks is usually modeled as a monolithic process in which reward prediction errors (RPEs) are used to update expected values of choice options. This modeling ignores the different contributions of different memory and decision-making systems thought to contribute even to simple learning. In an fMRI experiment, we investigated how working memory (WM) and incremental RL processes interact to guide human learning. WM load was manipulated by varying the number of stimuli to be learned across blocks. Behavioral results and computational modeling confirmed that learning was best explained as a mixture of two mechanisms: a fast, capacity-limited, and delay-sensitive WM process together with slower RL. Model-based analysis of fMRI data showed that striatum and lateral prefrontal cortex were sensitive to RPE, as shown previously, but, critically, these signals were reduced when the learning problem was within capacity of WM. The degree of this neural interaction related to individual differences in the use of WM to guide behavioral learning. These results indicate that the two systems do not process information independently, but rather interact during learning.SIGNIFICANCE STATEMENTReinforcement learning (RL) theory has been remarkably productive at improving our understanding of instrumental learning as well as dopaminergic and striatal network function across many mammalian species. However, this neural network is only one contributor to human learning and other mechanisms such as prefrontal cortex working memory also play a key role. Our results also show that these other players interact with the dopaminergic RL system, interfering with its key computation of reward prediction errors.
Department of Psychology and annecollins@berkeley.edu.|Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720, and.|Department of Cognitive, Linguistics, and Psychological Sciences, and.
Anne G E
0000-0003-3751-3662
AGE
Collins
Department of Cognitive, Linguistics, and Psychological Sciences, and.
Brittany
B
Ciullo
Department of Cognitive, Linguistics, and Psychological Sciences, and.|Brown Institute for Brain Science, Brown University, Providence, Rhode Island 02912.
Michael J
0000-0001-8451-0523
MJ
Frank
Department of Cognitive, Linguistics, and Psychological Sciences, and.|Brown Institute for Brain Science, Brown University, Providence, Rhode Island 02912.
David
D
Badre
D000293:Adolescent; D000328:Adult; D059031:Anticipation, Psychological; D003342:Corpus Striatum / Q000502:physiology; D005260:Female; D005625:Frontal Lobe / Q000502:physiology; D006801:Humans; D008297:Male; D008570:Memory, Short-Term; D011930:Reaction Time; D012201:Reward
fMRI; reinforcement learning; reward prediction error; working memory