title: Dissociable reward and timing signals in human midbrain and ventral striatum.
abstract: Reward prediction error (RPE) signals are central to current models of reward-learning. Temporal difference (TD) learning models posit that these signals should be modulated by predictions, not only of magnitude but also timing of reward. Here we show that BOLD activity in the VTA conforms to such TD predictions: responses to unexpected rewards are modulated by a temporal hazard function and activity between a predictive stimulus and reward is depressed in proportion to predicted reward. By contrast, BOLD activity in ventral striatum (VS) does not reflect a TD RPE, but instead encodes a signal on the variable relevant for behavior, here timing but not magnitude of reward. The results have important implications for dopaminergic models of cortico-striatal learning and suggest a modification of the conventional view that VS BOLD necessarily reflects inputs from dopaminergic VTA neurons signaling an RPE.
authors.affiliation: Sobell Department of Motor Neuroscience and Movement Disorders, Institute of Neurology, UCL, London WC1N3BG, UK. m.klein@ucl.ac.uk
authors.forename: Miriam C
authors.initials: MC
authors.lastname: Klein-Flügge
authors.forename: Laurence T
authors.initials: LT
authors.lastname: Hunt
authors.forename: Dominik R
authors.initials: DR
authors.lastname: Bach
authors.forename: Raymond J
authors.initials: RJ
authors.lastname: Dolan
authors.forename: Timothy E J
authors.initials: TE
authors.lastname: Behrens
mesh_terms: D000328:Adult; D001479:Basal Ganglia / Q000502:physiology*; D005260:Female; D006801:Humans; D008279:Magnetic Resonance Imaging / Q000379:methods; D008297:Male; D008636:Mesencephalon / Q000502:physiology*; D010775:Photic Stimulation / Q000379:methods; D011597:Psychomotor Performance / Q000502:physiology*; D012201:Reward; D013997:Time Factors; D055815:Young Adult
references.citation: J Neurosci. 2008 May 28;28(22):5623-30
references.pmid: 18509023
references.citation: Science. 2008 Feb 29;319(5867):1264-7
references.pmid: 18309087
references.citation: Nature. 2006 Aug 31;442(7106):1042-5
references.pmid: 16929307
references.citation: J Neurosci. 2007 May 2;27(18):4826-31
references.pmid: 17475790
references.citation: J Neurosci. 2010 Jan 6;30(1):47-55
references.pmid: 20053886
references.citation: IEEE Trans Med Imaging. 2004 Feb;23(2):137-52
references.pmid: 14964560
references.citation: Nature. 2008 Nov 13;456(7219):245-9
references.pmid: 19005555
references.citation: Magn Reson Med. 2000 Jul;44(1):162-7
references.pmid: 10893535
references.citation: J Neurosci. 2007 Nov 21;27(47):12860-7
references.pmid: 18032658
references.citation: J Neurosci. 2009 Oct 28;29(43):13524-31
references.pmid: 19864565
references.citation: J Neurosci. 2010 Mar 3;30(9):3210-9
references.pmid: 20203180
references.citation: Nat Neurosci. 2007 Dec;10(12):1615-24
references.pmid: 18026098
references.citation: Nat Neurosci. 2008 Aug;11(8):966-73
references.pmid: 18660807
references.citation: J Neurosci. 2002 Jun 1;22(11):4563-7
references.pmid: 12040063
references.citation: J Neurophysiol. 2006 Feb;95(2):948-59
references.pmid: 16192338
references.citation: Science. 2004 Apr 16;304(5669):452-4
references.pmid: 15087550
references.citation: Neuroimage. 2011 Jul 1;57(1):101-112
references.pmid: 21515386
references.citation: Neuroimage. 2006 Jul 15;31(4):1536-48
references.pmid: 16632379
references.citation: Soc Neurosci. 2010;5(1):76-91
references.pmid: 20119878
references.citation: Nat Neurosci. 2010 Jun;13(6):669-71
references.pmid: 20473290
references.citation: Med Image Anal. 2001 Jun;5(2):143-56
references.pmid: 11516708
references.citation: Hum Brain Mapp. 2002 Nov;17(3):143-55
references.pmid: 12391568
references.citation: Hum Brain Mapp. 2007 Apr;28(4):294-302
references.pmid: 16779798
references.citation: Neuroimage. 2004 Apr;21(4):1732-47
references.pmid: 15050594
references.citation: Proc Natl Acad Sci U S A. 2009 Oct 20;106(42):17951-6
references.pmid: 19822738
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):507-21
references.pmid: 12371508
references.citation: Nat Neurosci. 2007 Sep;10(9):1214-21
references.pmid: 17676057
references.citation: Curr Biol. 2010 Jul 13;20(13):1165-70
references.pmid: 20619815
references.citation: Neuroimage. 2010 Jan 1;49(1):772-81
references.pmid: 19682583
references.citation: Cereb Cortex. 2008 Jul;18(7):1485-95
references.pmid: 18033767
references.citation: Behav Brain Res. 2009 Apr 12;199(1):141-56
references.pmid: 18950662
references.citation: Neuroimage. 2004;23 Suppl 1:S208-19
references.pmid: 15501092
references.citation: Neuron. 2010 May 27;66(4):585-95
references.pmid: 20510862
references.citation: Neuroimage. 2002 Oct;17(2):825-41
references.pmid: 12377157
references.citation: Trends Neurosci. 2007 May;30(5):211-9
references.pmid: 17367873
references.citation: PLoS One. 2006 Dec 20;1:e103
references.pmid: 17183636
references.citation: Neuron. 2003 Apr 24;38(2):329-37
references.pmid: 12718865
references.citation: Cognition. 2009 Dec;113(3):262-280
references.pmid: 18926527
references.citation: Neuroimage. 2003 Oct;20(2):1052-63
references.pmid: 14568475
references.citation: J Neurosci. 2011 Apr 6;31(14):5504-11
references.pmid: 21471387
references.citation: Nat Neurosci. 1998 Aug;1(4):304-9
references.pmid: 10195164
references.citation: Nature. 2001 Jul 5;412(6842):43-8
references.pmid: 11452299
references.citation: J Neurophysiol. 2009 Dec;102(6):3384-91
references.pmid: 19793875
references.citation: J Neurosci. 2006 Sep 13;26(37):9530-7
references.pmid: 16971537
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347
references.citation: J Neurophysiol. 2006 Jan;95(1):301-10
references.pmid: 16192329
references.citation: Neural Comput. 2006 Jul;18(7):1637-77
references.pmid: 16764517
references.citation: Nature. 2004 Jun 10;429(6992):664-7
references.pmid: 15190354
references.citation: Neuroimage. 2010 Apr 1;50(2):644-56
references.pmid: 19969091
references.citation: Neuron. 2009 Jan 15;61(1):140-51
references.pmid: 19146819