title: Reinforcement learning with modulated spike timing dependent synaptic plasticity.
abstract: Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.
authors.affiliation: Department of Biology, University of Texas at San Antonio, San Antonio, TX 78249, USA. michael.farries@utsa.edu
authors.forename: Michael A
authors.initials: MA
authors.lastname: Farries
authors.forename: Adrienne L
authors.initials: AL
authors.lastname: Fairhall
mesh_terms: D000465:Algorithms; D004594:Electrophysiology; D007858:Learning / Q000502:physiology*; D008959:Models, Neurological; D016571:Neural Networks, Computer; D009473:Neuronal Plasticity / Q000502:physiology*; D018408:Patch-Clamp Techniques; D012054:Reinforcement, Psychology; D013569:Synapses / Q000502:physiology*