title: A spiking neural network model of an actor-critic learning agent.
abstract: The ability to adapt behavior to maximize reward as a result of interactions with the environment is crucial for the survival of any higher organism. In the framework of reinforcement learning, temporal-difference learning algorithms provide an effective strategy for such goal-directed adaptation, but it is unclear to what extent these algorithms are compatible with neural computation. In this article, we present a spiking neural network model that implements actor-critic temporal-difference learning by combining local plasticity rules with a global reward signal. The network is capable of solving a nontrivial gridworld task with sparse rewards. We derive a quantitative mapping of plasticity parameters and synaptic weights to the corresponding variables in the standard algorithmic formulation and demonstrate that the network learns with a similar speed to its discrete time counterpart and attains the same equilibrium performance.
authors.affiliation: Computational Neuroscience Group, RIKEN Brain Science Institute, Wako City, Saitama 351-0198, Japan. wiebke_potjans@brain.riken.jp
authors.forename: Wiebke
authors.initials: W
authors.lastname: Potjans
authors.forename: Abigail
authors.initials: A
authors.lastname: Morrison
authors.forename: Markus
authors.initials: M
authors.lastname: Diesmann
mesh_terms: D000200:Action Potentials / Q000502:physiology*; D000465:Algorithms; D000818:Animals; D003198:Computer Simulation; D006801:Humans; D007858:Learning / Q000502:physiology*; D008959:Models, Neurological; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*; D012054:Reinforcement, Psychology; D013997:Time Factors