title: The ubiquity of model-based reinforcement learning.
abstract: The reward prediction error (RPE) theory of dopamine (DA) function has enjoyed great success in the neuroscience of learning and decision-making. This theory is derived from model-free reinforcement learning (RL), in which choices are made simply on the basis of previously realized rewards. Recently, attention has turned to correlates of more flexible, albeit computationally complex, model-based methods in the brain. These methods are distinguished from model-free learning by their evaluation of candidate actions using expected future outcomes according to a world model. Puzzlingly, signatures from these computations seem to be pervasive in the very same regions previously thought to support model-free learning. Here, we review recent behavioral and neural evidence about these two systems, in attempt to reconcile their enigmatic cohabitation in the brain.
authors.affiliation: Department of Psychology, Columbia University, New York, NY, United States.
authors.forename: Bradley B
authors.initials: BB
authors.lastname: Doll
authors.forename: Dylan A
authors.initials: DA
authors.lastname: Simon
authors.forename: Nathaniel D
authors.initials: ND
authors.lastname: Daw
mesh_terms: D000818:Animals; D001921:Brain / Q000502:physiology*; D002755:Choice Behavior / Q000502:physiology; D003657:Decision Making / Q000502:physiology*; D006801:Humans; D008959:Models, Neurological; D012054:Reinforcement, Psychology; D012201:Reward