title: Vicarious reinforcement learning signals when instructing others.
abstract: Reinforcement learning (RL) theory posits that learning is driven by discrepancies between the predicted and actual outcomes of actions (prediction errors [PEs]). In social environments, learning is often guided by similar RL mechanisms. For example, teachers monitor the actions of students and provide feedback to them. This feedback evokes PEs in students that guide their learning. We report the first study that investigates the neural mechanisms that underpin RL signals in the brain of a teacher. Neurons in the anterior cingulate cortex (ACC) signal PEs when learning from the outcomes of one's own actions but also signal information when outcomes are received by others. Does a teacher's ACC signal PEs when monitoring a student's learning? Using fMRI, we studied brain activity in human subjects (teachers) as they taught a confederate (student) action-outcome associations by providing positive or negative feedback. We examined activity time-locked to the students' responses, when teachers infer student predictions and know actual outcomes. We fitted a RL-based computational model to the behavior of the student to characterize their learning, and examined whether a teacher's ACC signals when a student's predictions are wrong. In line with our hypothesis, activity in the teacher's ACC covaried with the PE values in the model. Additionally, activity in the teacher's insula and ventromedial prefrontal cortex covaried with the predicted value according to the student. Our findings highlight that the ACC signals PEs vicariously for others' erroneous predictions, when monitoring and instructing their learning. These results suggest that RL mechanisms, processed vicariously, may underpin and facilitate teaching behaviors.
authors.affiliation: Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford OX1 9DU, United Kingdom, Department of Experimental Psychology, University of Oxford, Oxford OX1 2JD, United Kingdom, Department of Psychology, Royal Holloway, University of London, Surrey TW20 0EX, United Kingdom, and matthew.apps@ndcn.ox.ac.uk.
authors.forename: Matthew A J
authors.initials: MA
authors.lastname: Apps
authors.affiliation: Department of Psychology, Royal Holloway, University of London, Surrey TW20 0EX, United Kingdom, and Neuroimaging Research Branch, Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Baltimore, Maryland 21224.
authors.forename: Elise
authors.initials: E
authors.lastname: Lesage
authors.affiliation: Department of Psychology, Royal Holloway, University of London, Surrey TW20 0EX, United Kingdom, and.
authors.forename: Narender
authors.initials: N
authors.lastname: Ramnani
mesh_terms: D000293:Adolescent; D000328:Adult; D001245:Association Learning / Q000502:physiology*; D001931:Brain Mapping; D003198:Computer Simulation; D003463:Cues; D030141:Feedback, Psychological / Q000502:physiology*; D005260:Female; D006179:Gyrus Cinguli / Q000098:blood supply / Q000502:physiology*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007206:Individuality; D008279:Magnetic Resonance Imaging; D008297:Male; D010100:Oxygen / Q000097:blood; D012054:Reinforcement, Psychology; D013663:Teaching; D055815:Young Adult
chemical_list: D010100:Oxygen
keywords: cingulate; fMRI; prediction error; reinforcement learning; social; teaching