title: Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans.
abstract: The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.
authors.affiliation: Neurology, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany; Bernstein Center for Computational Neuroscience, Charité - Universitätsmedizin Berlin, 10115 Berlin, Germany; Psychology, Humboldt Universität zu Berlin, 10099 Berlin, Germany; Computer Science, Chemnitz University of Technology, Chemnitz 09111, Germany. Electronic address: henning.schroll@informatik.tu-chemnitz.de.
authors.forename: Henning
authors.initials: H
authors.lastname: Schroll
authors.affiliation: Neurology, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany.
authors.forename: Andreas
authors.initials: A
authors.lastname: Horn
authors.affiliation: Neurology, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany.
authors.forename: Christine
authors.initials: C
authors.lastname: Gröschel
authors.affiliation: Neurology, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany.
authors.forename: Christof
authors.initials: C
authors.lastname: Brücke
authors.affiliation: Neurosurgery, Medical University Hanover, 30625 Hanover, Germany.
authors.forename: Götz
authors.initials: G
authors.lastname: Lütjens
authors.affiliation: Neurosurgery, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany.
authors.forename: Gerd-Helge
authors.initials: GH
authors.lastname: Schneider
authors.affiliation: Neurosurgery, Medical University Hanover, 30625 Hanover, Germany.
authors.forename: Joachim K
authors.initials: JK
authors.lastname: Krauss
authors.affiliation: Neurology, Charité - Universitätsmedizin Berlin, 13353 Berlin, Germany.
authors.forename: Andrea A
authors.initials: AA
authors.lastname: Kühn
authors.affiliation: Bernstein Center for Computational Neuroscience, Charité - Universitätsmedizin Berlin, 10115 Berlin, Germany; Computer Science, Chemnitz University of Technology, Chemnitz 09111, Germany. Electronic address: fred.hamker@informatik.tu-chemnitz.de.
authors.forename: Fred H
authors.initials: FH
authors.lastname: Hamker
mesh_terms: D000328:Adult; D000368:Aged; D001245:Association Learning / Q000502:physiology*; D058256:Brain Waves; D005260:Female; D005917:Globus Pallidus / Q000502:physiology*; D006801:Humans; D008297:Male; D008875:Middle Aged; D008959:Models, Neurological; D011597:Psychomotor Performance / Q000502:physiology*; D012201:Reward; D020651:Ventral Thalamic Nuclei / Q000502:physiology*; D055815:Young Adult
keywords: Basal ganglia; Exploitation; Exploration; Local field potential; Reinforcement learning; Reward