Can the apparent adaptation of dopamine neurons' mismatch sensitivities be reconciled with their computation of reward prediction errors?
According to modern reinforcement learning theories, midbrain dopamine (DA) neurons are part of an adaptive system within which learned expectations filter reward-related signals to enable computation of reward prediction errors (RPEs). Recent electrophysiological data on DA neuron responses to probabilistic reward schedules inspired the idea that DA neurons might be adapting their mismatch sensitivities to reflect variances of expected rewards. Taken literally as a mathematical hypothesis, this idea contradicts reinforcement learning theory, and most computational models of basal ganglia learning. Here, we report a qualitative mathematical derivation of the implications of a generic class of circuit models for learning to compute RPEs. This analysis and concordant circuit simulations, both of which predict DA neuron responses on probabilistic schedules, support a reinterpretation of the electrophysiological data that is fully compatible with the examined class of RPE models. This reinterpretation implies a novel and readily testable prediction.
Tan
Can Ozan
CO
Boston University, Cognitive and Neural Systems Department, Boston, MA 02215, USA.
Anderson
Eric
E
Dranias
Mark
M
Bullock
Daniel
D
D000200:Action Potentials / Q000502:physiology; D000818:Animals; D003198:Computer Simulation; D003463:Cues; D004298:Dopamine / Q000502:physiology*; D000882:Haplorhini / Q000502:physiology; D006801:Humans; D007858:Learning / Q000502:physiology; D015233:Models, Statistical; D009474:Neurons / Q000502:physiology*; D012054:Reinforcement, Psychology; D012201:Reward; D013378:Substantia Nigra / Q000502:physiology*; D009435:Synaptic Transmission / Q000502:physiology; D017557:Ventral Tegmental Area / Q000502:physiology*
D004298:Dopamine