title: Robust learning in SpikeProp.
abstract: Training a Spiking Neural Network using SpikeProp and its derivatives faces stability issues. Surges, marked by a sudden rise in learning cost, are a common occurrence during the learning process. They disrupt the learning process and often destabilize the process resulting in failure. A proper learning rate, which is neither too small nor too big, is important to minimize surges. Furthermore, external disturbances due to imperfection in sample data as well as internal disturbances are additional destabilizing source during the learning process. In this paper, we perform error system analysis incorporating external disturbance, followed by weight convergence analysis along with detailed robust stability analysis of SpikeProp learning process to ensure error bound of the learning process. Based on these results, we propose a robust adaptive learning rate scheme that aligns with the results of theoretical analysis. The performance of the proposed method has been compared with other prevalent methods based on different benchmark datasets and the results demonstrate that our method indeed has better performance in terms of convergence and learning speed as well.
authors.affiliation: School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798, Singapore. Electronic address: bam_sumit@hotmail.com.
authors.forename: Sumit Bam
authors.initials: SB
authors.lastname: Shrestha
authors.affiliation: School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798, Singapore. Electronic address: eqsong@ntu.edu.sg.
authors.forename: Qing
authors.initials: Q
authors.lastname: Song
mesh_terms: D000200:Action Potentials / Q000502:physiology; D000465:Algorithms; D006801:Humans; D000069550:Machine Learning / Q000639:trends; D016571:Neural Networks, Computer
keywords: Adaptive learning rate; Error analysis; Robust stability; Spiking Neural Network; Supervised learning; Weight convergence