title: fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning.
abstract: What are the neural dynamics of choice processes during reinforcement learning? Two largely separate literatures have examined dynamics of reinforcement learning (RL) as a function of experience but assuming a static choice process, or conversely, the dynamics of choice processes in decision making but based on static decision values. Here we show that human choice processes during RL are well described by a drift diffusion model (DDM) of decision making in which the learned trial-by-trial reward values are sequentially sampled, with a choice made when the value signal crosses a decision threshold. Moreover, simultaneous fMRI and EEG recordings revealed that this decision threshold is not fixed across trials but varies as a function of activity in the subthalamic nucleus (STN) and is further modulated by trial-by-trial measures of decision conflict and activity in the dorsomedial frontal cortex (pre-SMA BOLD and mediofrontal theta in EEG). These findings provide converging multimodal evidence for a model in which decision threshold in reward-based tasks is adjusted as a function of communication from pre-SMA to STN when choices differ subtly in reward values, allowing more time to choose the statistically more rewarding option.
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Brown Institute for Brain Science, Providence, Rhode Island 09212, Department of Psychiatry and Human Behavior, Brown University, Providence, Rhode Island 02912, michael_frank@brown.edu.
authors.forename: Michael J
authors.identifier: http://orcid.org/0000-0001-8451-0523
authors.initials: MJ
authors.lastname: Frank
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912.
authors.forename: Chris
authors.initials: C
authors.lastname: Gagne
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Department of Psychology and Program in Neuroscience, Bowdoin College, Brunswick, Maine 04011, and.
authors.forename: Erika
authors.initials: E
authors.lastname: Nyhus
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Brown Institute for Brain Science, Providence, Rhode Island 09212.
authors.forename: Sean
authors.initials: S
authors.lastname: Masters
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Brown Institute for Brain Science, Providence, Rhode Island 09212.
authors.forename: Thomas V
authors.initials: TV
authors.lastname: Wiecki
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Department of Psychology, University of New Mexico, Albuquerque, New Mexico 87131.
authors.forename: James F
authors.identifier: http://orcid.org/0000-0003-2428-3562
authors.initials: JF
authors.lastname: Cavanagh
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, Rhode Island 02912, Brown Institute for Brain Science, Providence, Rhode Island 09212.
authors.forename: David
authors.initials: D
authors.lastname: Badre
mesh_terms: D000293:Adolescent; D000328:Adult; D003213:Conditioning, Psychological; D003657:Decision Making; D004569:Electroencephalography; D005260:Female; D005625:Frontal Lobe / Q000502:physiology*; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D012054:Reinforcement, Psychology; D020531:Subthalamic Nucleus / Q000502:physiology*; D013826:Theta Rhythm
keywords: basal ganglia; decision making; drift diffusion model; prefrontal cortex; subthalamic nucleus