Multiple model-based reinforcement learning explains dopamine neuronal activity.
A number of computational models have explained the behavior of dopamine neurons in terms of temporal difference learning. However, earlier models cannot account for recent results of conditioning experiments; specifically, the behavior of dopamine neurons in case of variation of the interval between a cue stimulus and a reward has not been satisfyingly accounted for. We address this problem by using a modular architecture, in which each module consists of a reward predictor and a value estimator. A "responsibility signal", computed from the accuracy of the predictions of the reward predictors, is used to weight the contributions and learning of the value estimators. This multiple-model architecture gives an accurate account of the behavior of dopamine neurons in two specific experiments: when the reward is delivered earlier than expected, and when the stimulus-reward interval varies uniformly over a fixed range.
Bertin
Mathieu
M
ATR Computational Neuroscience Labs, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan. mbertin@atr.jp
Schweighofer
Nicolas
N
Doya
Kenji
K
D000818:Animals; D003198:Computer Simulation; D004298:Dopamine / Q000378:metabolism*; D008959:Models, Neurological; D009474:Neurons / Q000502:physiology*; D012054:Reinforcement, Psychology
D004298:Dopamine