Reinforcement learning with modulated spike timing dependent synaptic plasticity.
Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.
Farries
Michael A
MA
Department of Biology, University of Texas at San Antonio, San Antonio, TX 78249, USA. michael.farries@utsa.edu
Fairhall
Adrienne L
AL
D000465:Algorithms; D004594:Electrophysiology; D007858:Learning / Q000502:physiology*; D008959:Models, Neurological; D016571:Neural Networks, Computer; D009473:Neuronal Plasticity / Q000502:physiology*; D018408:Patch-Clamp Techniques; D012054:Reinforcement, Psychology; D013569:Synapses / Q000502:physiology*