Improved computation for Levenberg-Marquardt training.
The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.
Wilamowski
Bogdan M
BM
Department of Electrical and Computer Engineering, Auburn University, Auburn, AL 36849-5201, USA. wilam@ieee.org
Yu
Hao
H
D000465:Algorithms; D003198:Computer Simulation; D006801:Humans; D007858:Learning; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D013997:Time Factors