title: Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia.
abstract: Studies suggest that dopaminergic neurons report a unitary, global reward prediction error signal. However, learning in complex real-life tasks, in particular tasks that show hierarchical structure, requires multiple prediction errors that may coincide in time. We used functional neuroimaging to measure prediction error signals in humans performing such a hierarchical task involving simultaneous, uncorrelated prediction errors. Analysis of signals in a priori anatomical regions of interest in the ventral striatum and the ventral tegmental area indeed evidenced two simultaneous, but separable, prediction error signals corresponding to the two levels of hierarchy in the task. This result suggests that suitably designed tasks may reveal a more intricate pattern of firing in dopaminergic neurons. Moreover, the need for downstream separation of these signals implies possible limitations on the number of different task levels that we can learn about simultaneously.
authors.affiliation: Psychology Department and Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544, USA. cdiuk@princeton.edu
authors.forename: Carlos
authors.initials: C
authors.lastname: Diuk
authors.forename: Karin
authors.initials: K
authors.lastname: Tsai
authors.forename: Jonathan
authors.initials: J
authors.lastname: Wallis
authors.forename: Matthew
authors.initials: M
authors.lastname: Botvinick
authors.forename: Yael
authors.initials: Y
authors.lastname: Niv
mesh_terms: D000293:Adolescent; D000328:Adult; D001479:Basal Ganglia / Q000098:blood supply / Q000502:physiology*; D001931:Brain Mapping; D002755:Choice Behavior; D005260:Female; D006608:Hierarchy, Social; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007858:Learning / Q000502:physiology*; D016015:Logistic Models; D008279:Magnetic Resonance Imaging; D008297:Male; D016571:Neural Networks, Computer; D010100:Oxygen / Q000097:blood; D012054:Reinforcement, Psychology; D055815:Young Adult
chemical_list: D010100:Oxygen
references.citation: J Neurosci. 2008 May 28;28(22):5623-30
references.pmid: 18509023
references.citation: Science. 2008 Feb 29;319(5867):1264-7
references.pmid: 18309087
references.citation: Proc Natl Acad Sci U S A. 2011 Sep 13;108 Suppl 3:15647-54
references.pmid: 21389268
references.citation: Science. 2004 Apr 16;304(5669):452-4
references.pmid: 15087550
references.citation: Proc Natl Acad Sci U S A. 2009 Mar 24;106(12):4894-9
references.pmid: 19261850
references.citation: Nature. 2006 Jun 15;441(7095):876-9
references.pmid: 16778890
references.citation: Neuron. 2011 Nov 17;72(4):654-64
references.pmid: 22099466
references.citation: J Neurosci. 1996 Mar 1;16(5):1936-47
references.pmid: 8774460
references.citation: Nature. 2008 Nov 13;456(7219):245-9
references.pmid: 19005555
references.citation: Neuron. 2012 Jun 21;74(6):1125-37
references.pmid: 22726841
references.citation: Neuroimage. 2005 Feb 1;24(3):862-73
references.pmid: 15652321
references.citation: J Neurosci. 2007 Nov 21;27(47):12860-7
references.pmid: 18032658
references.citation: Spat Vis. 1997;10(4):433-6
references.pmid: 9176952
references.citation: Philos Trans R Soc Lond B Biol Sci. 2007 May 29;362(1481):933-42
references.pmid: 17395573
references.citation: Neuron. 2003 Apr 24;38(2):339-46
references.pmid: 12718866
references.citation: Neuroimage. 2006 Jun;31(2):790-5
references.pmid: 16487726
references.citation: Neuron. 2011 Jul 28;71(2):370-9
references.pmid: 21791294
references.citation: J Vis. 2011 Sep 16;11(10):10
references.pmid: 21926182
references.citation: Proc Natl Acad Sci U S A. 2007 May 29;104(22):9493-8
references.pmid: 17519340
references.citation: Neuron. 2010 May 27;66(4):585-95
references.pmid: 20510862
references.citation: Curr Opin Neurobiol. 2012 Dec;22(6):956-62
references.pmid: 22695048
references.citation: PLoS One. 2006 Dec 20;1:e103
references.pmid: 17183636
references.citation: Neuron. 2011 Mar 24;69(6):1204-15
references.pmid: 21435563
references.citation: Neuron. 2006 Aug 3;51(3):381-90
references.pmid: 16880132
references.citation: Neuron. 2003 Apr 24;38(2):329-37
references.pmid: 12718865
references.citation: Cognition. 2009 Dec;113(3):262-280
references.pmid: 18926527
references.citation: J Neurosci. 2011 Apr 6;31(14):5504-11
references.pmid: 21471387
references.citation: Proc Natl Acad Sci U S A. 2010 Aug 10;107(32):14431-6
references.pmid: 20660717
references.citation: J Neurosci. 2009 Oct 28;29(43):13524-31
references.pmid: 19864565
references.citation: J Neurosci. 2012 Jan 11;32(2):551-62
references.pmid: 22238090
references.citation: Neuron. 2002 Oct 10;36(2):241-63
references.pmid: 12383780
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347