Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates.
Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology, and machine learning to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.
Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ 08540, USA.
Alec
A
Solway
Matthew M
MM
Botvinick
D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D001499:Bayes Theorem; D001522:Behavior, Animal / Q000502:physiology; D001921:Brain / Q000502:physiology; D003198:Computer Simulation; D003213:Conditioning, Psychological / Q000502:physiology; D003657:Decision Making / Q000502:physiology*; D006040:Goals; D006184:Habits; D006801:Humans; D007858:Learning / Q000502:physiology*; D018782:Maze Learning; D008959:Models, Neurological; D008960:Models, Psychological; D009415:Nerve Net; D009488:Neurosciences; D011337:Probability Learning; D011340:Problem Solving; D051381:Rats; D012201:Reward