title: The anatomy of choice: dopamine and decision-making.
abstract: This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax choice rules and optimism bias emerge as natural consequences of free energy minimization. Previous accounts of active inference have focused on predictive coding. In this paper, we consider variational Bayes as a scheme that the brain might use for approximate Bayesian inference. This scheme provides formal constraints on the computational anatomy of inference and action, which appear to be remarkably consistent with neuroanatomy. Active inference contextualizes optimal decision theory within embodied inference, where goals become prior beliefs. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (associated with softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution. Crucially, this sensitivity corresponds to the precision of beliefs about behaviour. The changes in precision during variational updates are remarkably reminiscent of empirical dopaminergic responses-and they may provide a new perspective on the role of dopamine in assimilating reward prediction errors to optimize decision-making.
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK k.friston@ucl.ac.uk.
authors.forename: Karl
authors.initials: K
authors.lastname: Friston
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK.
authors.forename: Philipp
authors.initials: P
authors.lastname: Schwartenbeck
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK.
authors.forename: Thomas
authors.initials: T
authors.lastname: FitzGerald
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK.
authors.forename: Michael
authors.initials: M
authors.lastname: Moutoussis
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK Centre for Functional MRI of the Brain, The John Radcliffe Hospital, Headley Way, Oxford OX3 9DU, UK.
authors.forename: Timothy
authors.initials: T
authors.lastname: Behrens
authors.affiliation: The Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK.
authors.forename: Raymond J
authors.initials: RJ
authors.lastname: Dolan
mesh_terms: D001499:Bayes Theorem; D001921:Brain / Q000502:physiology*; D002755:Choice Behavior / Q000502:physiology*; D004298:Dopamine / Q000502:physiology*; D019277:Entropy; D006801:Humans; D008959:Models, Neurological
chemical_list: D004298:Dopamine
keywords: Bayesian inference; active inference; agency; bounded rationality; free energy; utility theory