Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways.
Midbrain dopamine neurons supposedly encode reward prediction error, but how error signals are computed remains elusive. Here, we propose a mechanism based on recent findings regarding corticostriatal circuits. Specifically, we propose that two distinct subpopulations of corticostriatal neurons differentially represent the animal's current and previous states/actions through unidirectional connectivity from one subpopulation to the other and strong recurrent excitation that exists only within the recipient subpopulation. These corticostriatal subpopulations selectively connect to the direct and indirect pathways of the basal ganglia, such that the temporal difference between the values of current and previous states/actions--the core of the error signal--can be computed. Our hypothesis suggests a unified view of basal ganglia functions and has important clinical implications.
Physical and Health Education, Graduate School of Education, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan. morita@p.u-tokyo.ac.jp
Kenji
K
Morita
Mieko
M
Morishima
Katsuyuki
K
Sakai
Yasuo
Y
Kawaguchi
D000818:Animals; D001921:Brain / Q000502:physiology*; D006801:Humans; D007858:Learning / Q000502:physiology*; D008959:Models, Neurological; D009434:Neural Pathways / Q000502:physiology*; D012054:Reinforcement, Psychology