title: Neural Mechanisms of Credit Assignment in a Multicue Environment.
abstract: In complex environments, many potential cues can guide a decision or be assigned responsibility for the outcome of the decision. We know little, however, about how humans and animals select relevant information sources that should guide behavior. We show that subjects solve this relevance selection and credit assignment problem by selecting one cue and its association with a particular outcome as the main focus of a hypothesis. To do this, we examined learning while using a task design that allowed us to estimate the focus of each subject's hypotheses on a trial-by-trial basis. When a prediction is confirmed by the outcome, then credit for the outcome is assigned to that cue rather than an alternative. Activity in medial frontal cortex is associated with the assignment of credit to the cue that is the main focus of the hypothesis. However, when the outcome disconfirms a prediction, the focus shifts between cues, and the credit for the outcome is assigned to an alternative cue. This process of reselection for credit assignment to an alternative cue is associated with lateral orbitofrontal cortex.
  
  
  CONCLUSIONS
  Learners should infer which features of environments are predictive of significant events, such as rewards. This "credit assignment" problem is particularly challenging when any of several cues might be predictive. We show that human subjects solve the credit assignment problem by implicitly "hypothesizing" which cue is relevant for predicting subsequent outcomes, and then credit is assigned according to this hypothesis. This process is associated with a distinctive pattern of activity in a part of medial frontal cortex. By contrast, when unexpected outcomes occur, hypotheses are redirected toward alternative cues, and this process is associated with activity in lateral orbitofrontal cortex.
authors.affiliation: Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford OX1 3UD, United Kingdom, Tokyo Metropolitan Institute of Medical Science, Setagaya-ku, Tokyo 156-8506, Japan, akaishirei@gmail.com.
authors.forename: Rei
authors.initials: R
authors.lastname: Akaishi
authors.affiliation: Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford OX1 3UD, United Kingdom.
authors.forename: Nils
authors.initials: N
authors.lastname: Kolling
authors.affiliation: Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana 47405, and.
authors.forename: Joshua W
authors.initials: JW
authors.lastname: Brown
authors.affiliation: Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford OX1 3UD, United Kingdom, Centre for Functional Magnetic Resonance Imaging of the Brain, University of Oxford, Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, Oxford OX3 9DU, United Kingdom.
authors.forename: Matthew
authors.initials: M
authors.lastname: Rushworth
mesh_terms: D000328:Adult; D003463:Cues; D003657:Decision Making / Q000502:physiology*; D004777:Environment; D005260:Female; D005625:Frontal Lobe / Q000098:blood supply / Q000502:physiology*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007858:Learning / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male; D008954:Models, Biological; D010100:Oxygen / Q000097:blood; D055815:Young Adult
chemical_list: D010100:Oxygen
keywords: decision making; learning; medial prefrontal cortex; orbitofrontal cortex
references.citation: Nat Neurosci. 2013 Apr;16(4):479-85
references.pmid: 23416449
references.citation: J Neurosci. 2012 Jul 25;32(30):10318-30
references.pmid: 22836265
references.citation: J Neurosci. 1997 Dec 1;17(23):9285-97
references.pmid: 9364074
references.citation: Neuron. 2010 Mar 25;65(6):927-39
references.pmid: 20346766
references.citation: Neuroimage. 2002 Oct;17(2):825-41
references.pmid: 12377157
references.citation: Annu Rev Neurosci. 2014;37:289-306
references.pmid: 25032497
references.citation: J Neurosci. 2003 Aug 27;23(21):7931-9
references.pmid: 12944524
references.citation: Nat Neurosci. 2015 Aug;18(8):1159-67
references.pmid: 26192748
references.citation: Science. 2014 Jun 27;344(6191):1481-6
references.pmid: 24876345
references.citation: Neuron. 2015 Feb 18;85(4):861-73
references.pmid: 25661183
references.citation: Nat Neurosci. 2001 Jan;4(1):95-102
references.pmid: 11135651
references.citation: Neuroimage. 2004;23 Suppl 1:S208-19
references.pmid: 15501092
references.citation: J Cogn Neurosci. 2014 Jul;26(7):1347-62
references.pmid: 24405106
references.citation: Brain. 2006 Apr;129(Pt 4):944-52
references.pmid: 16455794
references.citation: J Neurosci. 2010 Sep 15;30(37):12366-78
references.pmid: 20844132
references.citation: J Neurosci. 2011 Sep 14;31(37):13214-23
references.pmid: 21917804
references.citation: Nat Neurosci. 2011 Dec;14(12):1590-7
references.pmid: 22037501
references.citation: J Neurosci. 2009 Sep 16;29(37):11471-83
references.pmid: 19759296
references.citation: Proc Natl Acad Sci U S A. 2015 Apr 21;112(16):5195-200
references.pmid: 25848032
references.citation: Nat Neurosci. 1998 May;1(1):80-4
references.pmid: 10195114
references.citation: Neuron. 2009 Jun 11;62(5):733-43
references.pmid: 19524531
references.citation: Proc Natl Acad Sci U S A. 2006 Sep 12;103(37):13848-53
references.pmid: 16945915
references.citation: Annu Rev Neurosci. 2010;33:269-98
references.pmid: 20345247
references.citation: J Neurosci. 2012 Oct 3;32(40):13805-18
references.pmid: 23035092
references.citation: J Neurosci. 2011 Feb 16;31(7):2700-5
references.pmid: 21325538
references.citation: Nat Neurosci. 2014 Nov;17(11):1613-22
references.pmid: 25306549
references.citation: Elife. 2014;3:e02653
references.pmid: 25037263
references.citation: Med Image Anal. 2001 Jun;5(2):143-56
references.pmid: 11516708
references.citation: Front Hum Neurosci. 2012 Jan 24;5:189
references.pmid: 22291631
references.citation: J Neurosci. 2015 Sep 16;35(37):12813-23
references.pmid: 26377468
references.citation: Nat Neurosci. 2013 Aug;16(8):1140-5
references.pmid: 23792944
references.citation: Cereb Cortex. 2008 Jun;18(6):1374-83
references.pmid: 17928332
references.citation: Cereb Cortex. 2007 Jan;17(1):18-27
references.pmid: 16481566
references.citation: Neural Comput. 2010 Sep 1;22(9):2452-76
references.pmid: 20569174
references.citation: Prog Neurobiol. 2004 Apr;72(5):341-72
references.pmid: 15157726
references.citation: Nat Neurosci. 2007 Sep;10(9):1214-21
references.pmid: 17676057
references.citation: Brain. 2001 Feb;124(Pt 2):399-412
references.pmid: 11157567
references.citation: Neuron. 2011 May 26;70(4):731-41
references.pmid: 21609828
references.citation: J Neurosci. 2009 Jan 28;29(4):1175-90
references.pmid: 19176826
references.citation: Nature. 1996 Mar 7;380(6569):69-72
references.pmid: 8598908
references.citation: Neuron. 2015 Sep 2;87(5):1106-18
references.pmid: 26335649
references.citation: Eur J Neurosci. 2012 Apr;35(7):997-1010
references.pmid: 22487031
references.citation: Anim Learn Behav. 1976 May;4(2):186-92
references.pmid: 964444
references.citation: J Neurosci. 2015 May 27;35(21):8145-57
references.pmid: 26019331
references.citation: Neuron. 2011 Jun 23;70(6):1054-69
references.pmid: 21689594
references.citation: Nat Neurosci. 2014 Mar;17(3):463-70
references.pmid: 24509428
references.citation: J Neurosci. 2011 Oct 5;31(40):14399-412
references.pmid: 21976525
references.citation: Neuroimage. 2003 Oct;20(2):1052-63
references.pmid: 14568475
references.citation: Proc Natl Acad Sci U S A. 2010 Nov 23;107(47):20547-52
references.pmid: 21059901
references.citation: Proc Natl Acad Sci U S A. 2015 May 19;112(20):E2695-704
references.pmid: 25947150
references.citation: J Neurosci. 2013 Feb 13;33(7):3202-11
references.pmid: 23407973
references.citation: Nature. 2007 Jun 28;447(7148):1075-80
references.pmid: 17546027
references.citation: Hum Brain Mapp. 2002 Nov;17(3):143-55
references.pmid: 12391568
references.citation: Nature. 2007 Nov 1;450(7166):102-5
references.pmid: 17960136
references.citation: Neuron. 2014 Jun 18;82(6):1357-66
references.pmid: 24881835
references.citation: Neuroimage. 2004 Apr;21(4):1732-47
references.pmid: 15050594
references.citation: Annu Rev Neurosci. 2012;35:287-308
references.pmid: 22462543
references.citation: J Cereb Blood Flow Metab. 1992 Nov;12(6):900-18
references.pmid: 1400644
references.citation: Neuroimage. 2003 Jun;19(2 Pt 1):430-41
references.pmid: 12814592
references.citation: Science. 1996 Sep 6;273(5280):1399-402
references.pmid: 8703077