title: Global convergence of online BP training with dynamic learning rate.
abstract: The online backpropagation (BP) training procedure has been extensively explored in scientific research and engineering applications. One of the main factors affecting the performance of the online BP training is the learning rate. This paper proposes a new dynamic learning rate which is based on the estimate of the minimum error. The global convergence theory of the online BP training procedure with the proposed learning rate is further studied. It is proved that: 1) the error sequence converges to the global minimum error; and 2) the weight sequence converges to a fixed point at which the error function attains its global minimum. The obtained global convergence theory underlies the successful applications of the online BP training procedure. Illustrative examples are provided to support the theoretical analysis.
authors.forename: Rui
authors.initials: R
authors.lastname: Zhang
authors.forename: Zong-Ben
authors.initials: ZB
authors.lastname: Xu
authors.forename: Guang-Bin
authors.initials: GB
authors.lastname: Huang
authors.forename: Dianhui
authors.initials: D
authors.lastname: Wang
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D005246:Feedback; D015233:Models, Statistical; D016571:Neural Networks, Computer; D009862:Online Systems; D010363:Pattern Recognition, Automated / Q000379:methods*