A hypothesis for basal ganglia-dependent reinforcement learning in the songbird.
Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.
McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA. fee@mit.edu
M S
MS
Fee
J H
JH
Goldberg
D000161:Acoustic Stimulation; D000818:Animals; D001479:Basal Ganglia / Q000502:physiology*; D008954:Models, Biological; D009434:Neural Pathways / Q000502:physiology; D012054:Reinforcement, Psychology; D020308:Songbirds / Q000033:anatomy & histology* / Q000502:physiology*; D014828:Vocalization, Animal / Q000502:physiology*