title: Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates.
abstract: The mechanisms of reward maximization have been extensively studied at both the computational and neural levels. By contrast, little is known about how the brain learns to choose the options that minimize action cost. In principle, the brain could have evolved a general mechanism that applies the same learning rule to the different dimensions of choice options. To test this hypothesis, we scanned healthy human volunteers while they performed a probabilistic instrumental learning task that varied in both the physical effort and the monetary outcome associated with choice options. Behavioral data showed that the same computational rule, using prediction errors to update expectations, could account for both reward maximization and effort minimization. However, these learning-related variables were encoded in partially dissociable brain areas. In line with previous findings, the ventromedial prefrontal cortex was found to positively represent expected and actual rewards, regardless of effort. A separate network, encompassing the anterior insula, the dorsal anterior cingulate, and the posterior parietal cortex, correlated positively with expected and actual efforts. These findings suggest that the same computational rule is applied by distinct brain systems, depending on the choice dimension-cost or benefit-that has to be learned.
authors.affiliation: Motivation, Brain and Behavior Laboratory, Neuroimaging Research Center, Brain and Spine Institute, INSERM U975, CNRS UMR 7225, UPMC-P6 UMR S 1127, 7561 Paris Cedex 13, France.
authors.forename: Vasilisa
authors.initials: V
authors.lastname: Skvortsova
authors.affiliation: Motivation, Brain and Behavior Laboratory, Neuroimaging Research Center, Brain and Spine Institute, INSERM U975, CNRS UMR 7225, UPMC-P6 UMR S 1127, 7561 Paris Cedex 13, France, Laboratoire de Neurosciences Cognitives, INSERM U960, and Département d'Etudes Cognitives, Ecole Normale Supérieure, 7505, Paris, France.
authors.forename: Stefano
authors.initials: S
authors.lastname: Palminteri
authors.affiliation: Motivation, Brain and Behavior Laboratory, Neuroimaging Research Center, Brain and Spine Institute, INSERM U975, CNRS UMR 7225, UPMC-P6 UMR S 1127, 7561 Paris Cedex 13, France, mathias.pessiglione@gmail.com.
authors.forename: Mathias
authors.initials: M
authors.lastname: Pessiglione
mesh_terms: D000293:Adolescent; D000328:Adult; D002755:Choice Behavior; D003198:Computer Simulation; D003463:Cues; D005260:Female; D006801:Humans; D007858:Learning / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D005082:Physical Exertion / Q000502:physiology*; D012201:Reward; D055815:Young Adult
keywords: computational modeling; effort; reinforcement learning; reward; ventromedial prefrontal cortex