title: Model-based action planning involves cortico-cerebellar and basal ganglia networks.
abstract: Humans can select actions by learning, planning, or retrieving motor memories. Reinforcement Learning (RL) associates these processes with three major classes of strategies for action selection: exploratory RL learns state-action values by exploration, model-based RL uses internal models to simulate future states reached by hypothetical actions, and motor-memory RL selects past successful state-action mapping. In order to investigate the neural substrates that implement these strategies, we conducted a functional magnetic resonance imaging (fMRI) experiment while humans performed a sequential action selection task under conditions that promoted the use of a specific RL strategy. The ventromedial prefrontal cortex and ventral striatum increased activity in the exploratory condition; the dorsolateral prefrontal cortex, dorsomedial striatum, and lateral cerebellum in the model-based condition; and the supplementary motor area, putamen, and anterior cerebellum in the motor-memory condition. These findings suggest that a distinct prefrontal-basal ganglia and cerebellar network implements the model-based RL action selection strategy.
authors.affiliation: Graduate School of Information Science, Nara Institute of Science and Technology, Nara 630-0192, Japan.|Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan.|Brain Science Institute, Tamagawa University, Tokyo 194-8610, Japan.
authors.forename: Alan S R
authors.initials: AS
authors.lastname: Fermin
authors.affiliation: Graduate School of Information Science, Nara Institute of Science and Technology, Nara 630-0192, Japan.|Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan.
authors.forename: Takehiko
authors.initials: T
authors.lastname: Yoshida
authors.affiliation: Graduate School of Information Science, Nara Institute of Science and Technology, Nara 630-0192, Japan.|Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan.
authors.forename: Junichiro
authors.initials: J
authors.lastname: Yoshimoto
authors.affiliation: Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan.
authors.forename: Makoto
authors.initials: M
authors.lastname: Ito
authors.affiliation: ATR Brain Information Communication Research Lab, Kyoto 619-0288, Japan.
authors.forename: Saori C
authors.initials: SC
authors.lastname: Tanaka
authors.affiliation: Graduate School of Information Science, Nara Institute of Science and Technology, Nara 630-0192, Japan.|Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa 904-0495, Japan.|Brain Science Institute, Tamagawa University, Tokyo 194-8610, Japan.|ATR Brain Information Communication Research Lab, Kyoto 619-0288, Japan.
authors.forename: Kenji
authors.initials: K
authors.lastname: Doya
mesh_terms: D000328:Adult; D001479:Basal Ganglia / Q000502:physiology*; D001931:Brain Mapping / Q000379:methods*; D002525:Cerebellar Cortex / Q000502:physiology*; D005260:Female; D006801:Humans; D007858:Learning; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D011597:Psychomotor Performance; D055815:Young Adult