title: Rules and mechanisms for efficient two-stage learning in neural circuits.
abstract: Trial-and-error learning requires evaluating variable actions and reinforcing successful variants. In songbirds, vocal exploration is induced by LMAN, the output of a basal ganglia-related circuit that also contributes a corrective bias to the vocal output. This bias is gradually consolidated in RA, a motor cortex analogue downstream of LMAN. We develop a new model of such two-stage learning. Using stochastic gradient descent, we derive how the activity in 'tutor' circuits (e.g.,LMAN) should match plasticity mechanisms in 'student' circuits (e.g.,RA) to achieve efficient learning. We further describe a reinforcement learning framework through which the tutor can build its teaching signal. We show that mismatches between the tutor signal and the plasticity mechanism can impair learning. Applied to birdsong, our results predict the temporal structure of the corrective bias from LMAN given a plasticity rule in RA. Our framework can be applied predictively to other paired brain areas showing two-stage learning.
authors.affiliation: Initiative for the Theoretical Sciences, CUNY Graduate Center, New York, United States.|David Rittenhouse Laboratories, University of Pennsylvania, Philadelphia, United States.
authors.forename: Tiberiu
authors.identifier: 0000-0003-3107-3088
authors.initials: T
authors.lastname: Teşileanu
authors.affiliation: Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, United States.
authors.forename: Bence
authors.identifier: 0000-0003-2499-2705
authors.initials: B
authors.lastname: Ölveczky
authors.affiliation: Initiative for the Theoretical Sciences, CUNY Graduate Center, New York, United States.|David Rittenhouse Laboratories, University of Pennsylvania, Philadelphia, United States.|Theoretische Natuurkunde, Vrije Universiteit Brussel & International Solvay Institutes, Brussels, Belgium.
authors.forename: Vijay
authors.identifier: 0000-0002-6497-3819
authors.initials: V
authors.lastname: Balasubramanian
mesh_terms: D000818:Animals; D001479:Basal Ganglia / Q000502:physiology*; D046369:Finches; D007858:Learning; D008959:Models, Neurological; D009044:Motor Cortex / Q000502:physiology*; D009434:Neural Pathways / Q000502:physiology*; D009473:Neuronal Plasticity; D014828:Vocalization, Animal
keywords: birdsong; learning theory; motor control; neuroscience; reinforcement learning; zebra finch