Temporal difference models and reward-related learning in the human brain.
Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.
O'Doherty
John P
JP
Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, WC1N 3BG, London, United Kingdom. j.odoherty@fil.ion.ucl.ac.uk
Dayan
Peter
P
Friston
Karl
K
Critchley
Hugo
H
Dolan
Raymond J
RJ
D000293:Adolescent; D000328:Adult; D001921:Brain / Q000033:anatomy & histology / Q000502:physiology*; D001931:Brain Mapping; D003214:Conditioning, Classical / Q000502:physiology; D003342:Corpus Striatum / Q000033:anatomy & histology / Q000502:physiology; D005260:Female; D005625:Frontal Lobe / Q000033:anatomy & histology / Q000502:physiology; D006801:Humans; D007858:Learning / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male; D012016:Reference Values; D012025:Reflex, Pupillary / Q000502:physiology; D012201:Reward; D013649:Taste / Q000502:physiology; D013998:Time Perception / Q000502:physiology*