title: Model-based influences on humans' choices and striatal prediction errors.
abstract: The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.
authors: Center for Neural Science and Department of Psychology, New York University, New York, NY 10012, USA. daw@cns.nyu.edu
authors: Nathaniel D
authors: ND
authors: Daw
authors: Samuel J
authors: SJ
authors: Gershman
authors: Ben
authors: B
authors: Seymour
authors: Peter
authors: P
authors: Dayan
authors: Raymond J
authors: RJ
authors: Dolan
mesh_terms: D000328:Adult; D001479:Basal Ganglia / Q000502:physiology*; D001931:Brain Mapping; D002755:Choice Behavior / Q000502:physiology*; D004298:Dopamine / Q000378:metabolism*; D005260:Female; D006801:Humans; D016015:Logistic Models; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D009474:Neurons / Q000502:physiology; D009483:Neuropsychological Tests; D012054:Reinforcement, Psychology
chemical_list: D004298:Dopamine