title: Adaptive integration of habits into depth-limited planning defines a habitual-goal-directed spectrum.
abstract: Behavioral and neural evidence reveal a prospective goal-directed decision process that relies on mental simulation of the environment, and a retrospective habitual process that caches returns previously garnered from available choices. Artificial systems combine the two by simulating the environment up to some depth and then exploiting habitual values as proxies for consequences that may arise in the further future. Using a three-step task, we provide evidence that human subjects use such a normative plan-until-habit strategy, implying a spectrum of approaches that interpolates between habitual and goal-directed responding. We found that increasing time pressure led to shallower goal-directed planning, suggesting that a speed-accuracy tradeoff controls the depth of planning with deeper search leading to more accurate evaluation, at the cost of slower decision-making. We conclude that subjects integrate habit-based cached values directly into goal-directed evaluations in a normative manner.
authors: Gatsby Computational Neuroscience Unit, University College London, London W1T 4JG, United Kingdom; mehdi@gatsby.ucl.ac.uk.
authors: Mehdi
authors: M
authors: Keramati
authors: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, United Kingdom.
authors: Peter
authors: P
authors: Smittenaar
authors: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London WC1N 3BG, United Kingdom.|Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom.
authors: Raymond J
authors: RJ
authors: Dolan
authors: Gatsby Computational Neuroscience Unit, University College London, London W1T 4JG, United Kingdom.|Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom.
authors: Peter
authors: P
authors: Dayan
keywords: habit; planning; reinforcement learning; speed/accuracy tradeoff; tree-based evaluation