title: Reducing the computational footprint for real-time BCPNN learning.
abstract: The implementation of synaptic plasticity in neural simulation or neuromorphic hardware is usually very resource-intensive, often requiring a compromise between efficiency and flexibility. A versatile, but computationally-expensive plasticity mechanism is provided by the Bayesian Confidence Propagation Neural Network (BCPNN) paradigm. Building upon Bayesian statistics, and having clear links to biological plasticity processes, the BCPNN learning rule has been applied in many fields, ranging from data classification, associative memory, reward-based learning, probabilistic inference to cortical attractor memory networks. In the spike-based version of this learning rule the pre-, postsynaptic and coincident activity is traced in three low-pass-filtering stages, requiring a total of eight state variables, whose dynamics are typically simulated with the fixed step size Euler method. We derive analytic solutions allowing an efficient event-driven implementation of this learning rule. Further speedup is achieved by first rewriting the model which reduces the number of basic arithmetic operations per update to one half, and second by using look-up tables for the frequently calculated exponential decay. Ultimately, in a typical use case, the simulation using our approach is more than one order of magnitude faster than with the fixed step size Euler method. Aiming for a small memory footprint per BCPNN synapse, we also evaluate the use of fixed-point numbers for the state variables, and assess the number of bits required to achieve same or better accuracy than with the conventional explicit Euler method. All of this will allow a real-time simulation of a reduced cortex model based on BCPNN in high performance computing. More important, with the analytic solution at hand and due to the reduced memory bandwidth, the learning rule can be efficiently implemented in dedicated or existing digital neuromorphic hardware.
authors: Department of Electrical Engineering and Information Technology, Technische Universität Dresden Germany.
authors: Bernhard
authors: B
authors: Vogginger
authors: Department of Electrical Engineering and Information Technology, Technische Universität Dresden Germany.
authors: René
authors: R
authors: Schüffny
authors: Department of Computational Biology, School of Computer Science and Communication, Royal Institute of Technology (KTH) Stockholm, Sweden ; Department of Numerical Analysis and Computer Science, Stockholm University Stockholm, Sweden.
authors: Anders
authors: A
authors: Lansner
authors: Department of Electrical Engineering and Information Technology, Technische Universität Dresden Germany.
authors: Love
authors: L
authors: Cederström
authors: Department of Electrical Engineering and Information Technology, Technische Universität Dresden Germany.
authors: Johannes
authors: J
authors: Partzsch
authors: Department of Electrical Engineering and Information Technology, Technische Universität Dresden Germany.
authors: Sebastian
authors: S
authors: Höppner
keywords: Bayesian confidence propagation neural network (BCPNN); Hebbian learning; digital neuromorphic hardware; event-driven simulation; fixed-point accuracy; look-up tables; spiking neural networks; synaptic plasticity
references: Neural Comput. 2006 Dec;18(12):2959-93
references: 17052155
references: Front Synaptic Neurosci. 2014 Apr 08;6:8
references: 24782758
references: J Neurosci. 2013 Jul 17;33(29):11817-24
references: 23864671
references: Eur J Clin Pharmacol. 1998 Jun;54(4):315-21
references: 9696956
references: J Cogn Neurosci. 2011 Oct;23(10):3008-20
references: 21452933
references: J Comput Neurosci. 2007 Dec;23(3):349-98
references: 17629781
references: Neural Comput. 2006 Aug;18(8):2004-27
references: 16771661
references: Front Behav Neurosci. 2012 Oct 02;6:65
references: 23060764
references: Front Neural Circuits. 2014 Feb 07;8:5
references: 24570657
references: Biol Cybern. 2008 Jun;98(6):459-78
references: 18491160
references: J Neurosci. 2007 Nov 7;27(45):12242-54
references: 17989290
references: Drug Saf. 2000 Dec;23(6):533-42
references: 11144660
references: Neural Comput. 2003 Jan;15(1):67-101
references: 12590820
references: Int J Neural Syst. 1996 May;7(2):115-28
references: 8823623
references: Front Neurosci. 2011 May 31;5:73
references: 21747754
references: PLoS One. 2013 Aug 30;8(8):e73776
references: 24023690
references: Eur J Neurosci. 1994 Jun 1;6(6):898-917
references: 7952278
references: Front Comput Neurosci. 2011 May 03;5:13
references: 21625630
references: Network. 2013;24(4):129-50
references: 24251411
references: Neural Netw. 2007 Jan;20(1):48-61
references: 16860539
references: Curr Biol. 2003 Mar 18;13(6):493-7
references: 12646132
references: Science. 2014 Aug 8;345(6197):668-73
references: 25104385
references: Neural Netw. 2013 Sep;45:4-26
references: 23886551
references: Front Comput Neurosci. 2014 Jul 01;8:64
references: 25071536
references: Front Neurosci. 2013 Sep 10;7:118
references: 24058330
references: J Comput Neurosci. 2012 Apr;32(2):309-26
references: 21837455
references: J Neurosci. 1998 Dec 15;18(24):10464-72
references: 9852584
references: Network. 2003 Nov;14(4):789-802
references: 14653503
references: PLoS Comput Biol. 2010 Jun 03;6(6):e1000803
references: 20532199
references: Trends Neurosci. 2009 Mar;32(3):178-86
references: 19187979