title: Interplay of approximate planning strategies.
abstract: Humans routinely formulate plans in domains so complex that even the most powerful computers are taxed. To do so, they seem to avail themselves of many strategies and heuristics that efficiently simplify, approximate, and hierarchically decompose hard tasks into simpler subtasks. Theoretical and cognitive research has revealed several such strategies; however, little is known about their establishment, interaction, and efficiency. Here, we use model-based behavioral analysis to provide a detailed examination of the performance of human subjects in a moderately deep planning task. We find that subjects exploit the structure of the domain to establish subgoals in a way that achieves a nearly maximal reduction in the cost of computing values of choices, but then combine partial searches with greedy local steps to solve subtasks, and maladaptively prune the decision trees of subtasks in a reflexive manner upon encountering salient losses. Subjects come idiosyncratically to favor particular sequences of actions to achieve subgoals, creating novel complex actions or "options."
authors: Translational Neuromodeling Unit, Institute of Biomedical Engineering, University of Zürich and Swiss Federal Institute of Technology (ETH) Zürich, 8032 Zurich, Switzerland; Department of Psychiatry, Psychotherapy and Psychosomatics, Hospital of Psychiatry, University of Zürich, 8032 Zurich, Switzerland; qhuys@cantab.net.
authors: Quentin J M
authors: QJ
authors: Huys
authors: Institute of Cognitive Neuroscience, University College London, London WC1N 3AR, United Kingdom; Experimental Therapeutics & Pathophysiology Branch, Intramural Research Program, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892;
authors: Níall
authors: N
authors: Lally
authors: Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, CA 90095;
authors: Paul
authors: P
authors: Faulkner
authors: Program in Neuroscience and MD-PhD Program, Harvard Medical School, Boston, MA 02115;
authors: Neir
authors: N
authors: Eshel
authors: Department of Psychiatry, Psychotherapy and Psychosomatics, Hospital of Psychiatry, University of Zürich, 8032 Zurich, Switzerland;
authors: Erich
authors: E
authors: Seifritz
authors: Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139; and.
authors: Samuel J
authors: SJ
authors: Gershman
authors: Gatsby Computational Neuroscience Unit, University College London, London WC1N 3AR, United Kingdom.
authors: Peter
authors: P
authors: Dayan
authors: Institute of Cognitive Neuroscience, University College London, London WC1N 3AR, United Kingdom;
authors: Jonathan P
authors: JP
authors: Roiser
mesh_terms: D006801:Humans; D007360:Intelligence; D010934:Planning Techniques; D013269:Stochastic Processes
keywords: hierarchical reinforcement learning; memoization; planning; pruning