title: Multiple model-based reinforcement learning explains dopamine neuronal activity.
abstract: A number of computational models have explained the behavior of dopamine neurons in terms of temporal difference learning. However, earlier models cannot account for recent results of conditioning experiments; specifically, the behavior of dopamine neurons in case of variation of the interval between a cue stimulus and a reward has not been satisfyingly accounted for. We address this problem by using a modular architecture, in which each module consists of a reward predictor and a value estimator. A "responsibility signal", computed from the accuracy of the predictions of the reward predictors, is used to weight the contributions and learning of the value estimators. This multiple-model architecture gives an accurate account of the behavior of dopamine neurons in two specific experiments: when the reward is delivered earlier than expected, and when the stimulus-reward interval varies uniformly over a fixed range.
authors.affiliation: ATR Computational Neuroscience Labs, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan. mbertin@atr.jp
authors.forename: Mathieu
authors.initials: M
authors.lastname: Bertin
authors.forename: Nicolas
authors.initials: N
authors.lastname: Schweighofer
authors.forename: Kenji
authors.initials: K
authors.lastname: Doya
mesh_terms: D000818:Animals; D003198:Computer Simulation; D004298:Dopamine / Q000378:metabolism*; D008959:Models, Neurological; D009474:Neurons / Q000502:physiology*; D012054:Reinforcement, Psychology
chemical_list: D004298:Dopamine