title: The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive.
abstract: A number of accounts of human and animal behavior posit the operation of parallel and competing valuation systems in the control of choice behavior. In these accounts, a flexible but computationally expensive model-based reinforcement-learning system has been contrasted with a less flexible but more efficient model-free reinforcement-learning system. The factors governing which system controls behavior-and under what circumstances-are still unclear. Following the hypothesis that model-based reinforcement learning requires cognitive resources, we demonstrated that having human decision makers perform a demanding secondary task engenders increased reliance on a model-free reinforcement-learning strategy. Further, we showed that, across trials, people negotiate the trade-off between the two systems dynamically as a function of concurrent executive-function demands, and people's choice latencies reflect the computational expenses of the strategy they employ. These results demonstrate that competition between multiple learning systems can be controlled on a trial-by-trial basis by modulating the availability of cognitive resources.
authors.affiliation: Department of Psychology, University of Texas at Austin, USA. rotto@nyu.edu
authors.forename: A Ross
authors.initials: AR
authors.lastname: Otto
authors.forename: Samuel J
authors.initials: SJ
authors.lastname: Gershman
authors.forename: Arthur B
authors.initials: AB
authors.lastname: Markman
authors.forename: Nathaniel D
authors.initials: ND
authors.lastname: Daw
mesh_terms: D002755:Choice Behavior / Q000502:physiology*; D003463:Cues; D003657:Decision Making / Q000502:physiology*; D006801:Humans; D012054:Reinforcement, Psychology; D013334:Students / Q000523:psychology; D013647:Task Performance and Analysis
keywords: cognitive neuroscience; decision making
references.citation: Science. 2004 Oct 15;306(5695):503-7
references.pmid: 15486304
references.citation: Trends Cogn Sci. 2003 Dec;7(12):547-52
references.pmid: 14643371
references.citation: PLoS Comput Biol. 2011 May;7(5):e1002055
references.pmid: 21637741
references.citation: J Neurosci. 1996 Mar 1;16(5):1936-47
references.pmid: 8774460
references.citation: Cognition. 2011 Feb;118(2):274-9
references.pmid: 21145046
references.citation: Mem Cognit. 2006 Mar;34(2):387-98
references.pmid: 16752602
references.citation: Neuron. 2010 May 27;66(4):585-95
references.pmid: 20510862
references.citation: Psychon Bull Rev. 2001 Mar;8(1):168-76
references.pmid: 11340863
references.citation: Nat Neurosci. 2005 Nov;8(11):1481-9
references.pmid: 16251991
references.citation: Psychol Rev. 1999 Jan;106(1):3-19
references.pmid: 10197361
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347
references.citation: Neural Netw. 2009 Apr;22(3):213-9
references.pmid: 19362448
references.citation: Cogn Psychol. 2000 Aug;41(1):49-100
references.pmid: 10945922
references.citation: Mem Cognit. 2007 Jul;35(5):864-74
references.pmid: 17910172
references.citation: Nat Rev Neurosci. 2006 Jun;7(6):464-76
references.pmid: 16715055
references.citation: Neuron. 2011 Mar 24;69(6):1204-15
references.pmid: 21435563
references.citation: J Neurosci. 2007 Apr 11;27(15):4019-26
references.pmid: 17428979
references.citation: Nat Neurosci. 2005 Dec;8(12):1704-11
references.pmid: 16286932
references.citation: Neuron. 2003 Apr 24;38(2):329-37
references.pmid: 12718865
references.citation: Proc Natl Acad Sci U S A. 2006 Aug 1;103(31):11778-83
references.pmid: 16868087