title: Event-driven contrastive divergence for spiking neuromorphic systems.
abstract: Restricted Boltzmann Machines (RBMs) and Deep Belief Networks have been demonstrated to perform efficiently in a variety of applications, such as dimensionality reduction, feature learning, and classification. Their implementation on neuromorphic hardware platforms emulating large-scale networks of spiking neurons can have significant advantages from the perspectives of scalability, power dissipation and real-time interfacing with the environment. However, the traditional RBM architecture and the commonly used training algorithm known as Contrastive Divergence (CD) are based on discrete updates and exact arithmetics which do not directly map onto a dynamical neural substrate. Here, we present an event-driven variation of CD to train a RBM constructed with Integrate & Fire (I&F) neurons, that is constrained by the limitations of existing and near future neuromorphic hardware platforms. Our strategy is based on neural sampling, which allows us to synthesize a spiking neural network that samples from a target Boltzmann distribution. The recurrent activity of the network replaces the discrete steps of the CD algorithm, while Spike Time Dependent Plasticity (STDP) carries out the weight updates in an online, asynchronous fashion. We demonstrate our approach by training an RBM composed of leaky I&F neurons with STDP synapses to learn a generative model of the MNIST hand-written digit dataset, and by testing it in recognition, generation and cue integration tasks. Our results contribute to a machine learning-driven approach for synthesizing networks of spiking neurons capable of carrying out practical, high-level functionality.
authors.affiliation: Institute for Neural Computation, University of California San Diego, La Jolla, CA, USA.
authors.forename: Emre
authors.initials: E
authors.lastname: Neftci
authors.affiliation: Institute for Neural Computation, University of California San Diego, La Jolla, CA, USA ; Electrical and Computer Engineering Department, University of California San Diego, La Jolla, CA, USA.
authors.forename: Srinjoy
authors.initials: S
authors.lastname: Das
authors.affiliation: Department of Bioengineering, University of California San Diego, La Jolla, CA, USA.
authors.forename: Bruno
authors.initials: B
authors.lastname: Pedroni
authors.affiliation: Institute for Neural Computation, University of California San Diego, La Jolla, CA, USA ; Electrical and Computer Engineering Department, University of California San Diego, La Jolla, CA, USA.
authors.forename: Kenneth
authors.initials: K
authors.lastname: Kreutz-Delgado
authors.affiliation: Institute for Neural Computation, University of California San Diego, La Jolla, CA, USA ; Department of Bioengineering, University of California San Diego, La Jolla, CA, USA.
authors.forename: Gert
authors.initials: G
authors.lastname: Cauwenberghs
keywords: Markov chain monte carlo; generative model; neuromorphic cognition; recurrent neural network; synaptic plasticity