title: Model-Based Reasoning in Humans Becomes Automatic with Training.
abstract: Model-based and model-free reinforcement learning (RL) have been suggested as algorithmic realizations of goal-directed and habitual action strategies. Model-based RL is more flexible than model-free but requires sophisticated calculations using a learnt model of the world. This has led model-based RL to be identified with slow, deliberative processing, and model-free RL with fast, automatic processing. In support of this distinction, it has recently been shown that model-based reasoning is impaired by placing subjects under cognitive load--a hallmark of non-automaticity. Here, using the same task, we show that cognitive load does not impair model-based reasoning if subjects receive prior training on the task. This finding is replicated across two studies and a variety of analysis methods. Thus, task familiarity permits use of model-based reasoning in parallel with other cognitive demands. The ability to deploy model-based reasoning in an automatic, parallelizable fashion has widespread theoretical implications, particularly for the learning and execution of complex behaviors. It also suggests a range of important failure modes in psychiatric disorders.
authors.affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom.
authors.forename: Marcos
authors.initials: M
authors.lastname: Economides
authors.affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom; Max Planck Centre for Computational Psychiatry and Ageing, University College London, London, United Kingdom.
authors.forename: Zeb
authors.initials: Z
authors.lastname: Kurth-Nelson
authors.affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom.
authors.forename: Annika
authors.initials: A
authors.lastname: Lübbert
authors.affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom; Ageing Research Centre, Karolinska Institute, Stockholm, Sweden.
authors.forename: Marc
authors.initials: M
authors.lastname: Guitart-Masip
authors.affiliation: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, United Kingdom; Max Planck Centre for Computational Psychiatry and Ageing, University College London, London, United Kingdom.
authors.forename: Raymond J
authors.initials: RJ
authors.lastname: Dolan
mesh_terms: D000293:Adolescent; D000328:Adult; D000465:Algorithms; D019295:Computational Biology; D003657:Decision Making / Q000502:physiology*; D005260:Female; D006801:Humans; D008297:Male; D008954:Models, Biological; D012054:Reinforcement, Psychology; D012201:Reward; D013647:Task Performance and Analysis; D055815:Young Adult