title: The involvement of model-based but not model-free learning signals during observational reward learning in the absence of choice.
abstract: A major open question is whether computational strategies thought to be used during experiential learning, specifically model-based and model-free reinforcement learning, also support observational learning. Furthermore, the question of how observational learning occurs when observers must learn about the value of options from observing outcomes in the absence of choice has not been addressed. In the present study we used a multi-armed bandit task that encouraged human participants to employ both experiential and observational learning while they underwent functional magnetic resonance imaging (fMRI). We found evidence for the presence of model-based learning signals during both observational and experiential learning in the intraparietal sulcus. However, unlike during experiential learning, model-free learning signals in the ventral striatum were not detectable during this form of observational learning. These results provide insight into the flexibility of the model-based learning system, implicating this system in learning during observation as well as from direct experience, and further suggest that the model-free reinforcement learning system may be less flexible with regard to its involvement in observational learning.
authors.affiliation: Trinity College Institute of Neuroscience, Trinity College, Dublin, Ireland; Computation and Neural Systems Program, California Institute of Technology, Pasadena, California; sdunne@caltech.edu.
authors.forename: Simon
authors.initials: S
authors.lastname: Dunne
authors.affiliation: Trinity College Institute of Neuroscience, Trinity College, Dublin, Ireland; Department of Psychology, University of Freiburg, Freiburg, Germany; and.
authors.forename: Arun
authors.initials: A
authors.lastname: D'Souza
authors.affiliation: Trinity College Institute of Neuroscience, Trinity College, Dublin, Ireland; Computation and Neural Systems Program, California Institute of Technology, Pasadena, California; Division of Humanities and Social Sciences, California Institute of Technology.
authors.forename: John P
authors.initials: JP
authors.lastname: O'Doherty
mesh_terms: D000328:Adult; D002755:Choice Behavior / Q000502:physiology*; D003463:Cues; D005260:Female; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007858:Learning / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D010100:Oxygen / Q000097:blood; D010296:Parietal Lobe / Q000000981:diagnostic imaging / Q000502:physiology*; D011597:Psychomotor Performance / Q000502:physiology*; D012201:Reward; D055815:Young Adult
chemical_list: D010100:Oxygen
keywords: fMRI; observational learning; reinforcement learning; reward prediction error; ventral striatum