title: Convergence study in extended Kalman filter-based training of recurrent neural networks.
abstract: Recurrent neural network (RNN) has emerged as a promising tool in modeling nonlinear dynamical systems, but the training convergence is still of concern. This paper aims to develop an effective extended Kalman filter-based RNN training approach with a controllable training convergence. The training convergence problem during extended Kalman filter-based RNN training has been proposed and studied by adapting two artificial training noise parameters: the covariance of measurement noise (R) and the covariance of process noise (Q) of Kalman filter. The R and Q adaption laws have been developed using the Lyapunov method and the maximum likelihood method, respectively. The effectiveness of the proposed adaption laws has been tested using a nonlinear dynamical benchmark system and further applied in cutting tool wear modeling. The results show that the R adaption law can effectively avoid the divergence problem and ensure the training convergence, whereas the Q adaption law helps improve the training convergence speed.
authors.affiliation: Clemson University, Clemson, SC 29630 USA. xywang@ufl.edu
authors.forename: Xiaoyu
authors.initials: X
authors.lastname: Wang
authors.forename: Yong
authors.initials: Y
authors.lastname: Huang
mesh_terms: D001185:Artificial Intelligence; D003198:Computer Simulation; D006801:Humans; D007858:Learning / Q000502:physiology*; D016013:Likelihood Functions; D016571:Neural Networks, Computer; D017711:Nonlinear Dynamics; D012008:Recurrence