title: Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis.
abstract: Reinforcement learning describes motivated behavior in terms of two abstract signals. The representation of discrepancies between expected and actual rewards/punishments-prediction error-is thought to update the expected value of actions and predictive stimuli. Electrophysiological and lesion studies have suggested that mesostriatal prediction error signals control behavior through synaptic modification of cortico-striato-thalamic networks. Signals in the ventromedial prefrontal and orbitofrontal cortex are implicated in representing expected value. To obtain unbiased maps of these representations in the human brain, we performed a meta-analysis of functional magnetic resonance imaging studies that had employed algorithmic reinforcement learning models across a variety of experimental paradigms. We found that the ventral striatum (medial and lateral) and midbrain/thalamus represented reward prediction errors, consistent with animal studies. Prediction error signals were also seen in the frontal operculum/insula, particularly for social rewards. In Pavlovian studies, striatal prediction error signals extended into the amygdala, whereas instrumental tasks engaged the caudate. Prediction error maps were sensitive to the model-fitting procedure (fixed or individually estimated) and to the extent of spatial smoothing. A correlate of expected value was found in a posterior region of the ventromedial prefrontal cortex, caudal and medial to the orbitofrontal regions identified in animal studies. These findings highlight a reproducible motif of reinforcement learning in the cortico-striatal loops and identify methodological dimensions that may influence the reproducibility of activation patterns across studies.
authors: Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA, chaseh@upmc.edu.
authors: Henry W
authors: HW
authors: Chase
authors: Poornima
authors: P
authors: Kumar
authors: Simon B
authors: SB
authors: Eickhoff
authors: Alexandre Y
authors: AY
authors: Dombrovski
mesh_terms: D000818:Animals; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping; D006801:Humans; D016013:Likelihood Functions; D008960:Models, Psychological; D012054:Reinforcement, Psychology; D013223:Statistics as Topic