title: Deterministic convergence of chaos injection-based gradient method for training feedforward neural networks.
abstract: It has been shown that, by adding a chaotic sequence to the weight update during the training of neural networks, the chaos injection-based gradient method (CIBGM) is superior to the standard backpropagation algorithm. This paper presents the theoretical convergence analysis of CIBGM for training feedforward neural networks. We consider both the case of batch learning as well as the case of online learning. Under mild conditions, we prove the weak convergence, i.e., the training error tends to a constant and the gradient of the error function tends to zero. Moreover, the strong convergence of CIBGM is also obtained with the help of an extra condition. The theoretical results are substantiated by a simulation example.
authors.affiliation: Department of Mathematics, Dalian Maritime University, Dalian, 116026 People's Republic of China ; Research Center of Information and Control, Dalian University of Technology, Dalian, 116024 People's Republic of China.
authors.forename: Huisheng
authors.initials: H
authors.lastname: Zhang
authors.affiliation: Department of Mathematics, Dalian Maritime University, Dalian, 116026 People's Republic of China.
authors.forename: Ying
authors.initials: Y
authors.lastname: Zhang
authors.affiliation: College of Science, Harbin Engineering University, Harbin, 150001 People's Republic of China.
authors.forename: Dongpo
authors.initials: D
authors.lastname: Xu
authors.affiliation: Research Center of Information and Control, Dalian University of Technology, Dalian, 116024 People's Republic of China.
authors.forename: Xiaodong
authors.initials: X
authors.lastname: Liu
keywords: Batch learning; Chaos injection-based gradient method; Convergence; Feedforward neural networks; Online learning