title: Robust initialization of a Jordan network with recurrent constrained learning.
abstract: In this paper, we propose a robust initialization of a Jordan network with a recurrent constrained learning (RIJNRCL) algorithm for multilayered recurrent neural networks (RNNs). This novel algorithm is based on the constrained learning concept of the Jordan network with a recurrent sensitivity and weight convergence analysis, which is used to obtain a tradeoff between the training and testing errors. In addition to using classical techniques for the adaptive learning rate and the adaptive dead zone, RIJNRCL employs a recurrent constrained parameter matrix to switch off excessive contributions from the hidden layer neurons based on weight convergence and stability conditions of the multilayered RNNs. It is well known that a good response from the hidden layer neurons and proper initialization play a dominant role in avoiding local minima in multilayered RNNs. The new RIJNRCL algorithm solves the twin problems of weight initialization and selection of the hidden layer neurons via a novel recurrent sensitivity ratio analysis. We provide the detailed steps for using RIJNRCL in a few benchmark time-series prediction problems and show that the proposed algorithm achieves superior generalization performance.
authors.affiliation: School of Electrical and Electronic Engineering, Nanyang Technological University, 639798 Singapore. eqsong@ntu.edu.sg
authors.forename: Qing
authors.initials: Q
authors.lastname: Song
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D008962:Models, Theoretical; D016571:Neural Networks, Computer