title: Improved computation for Levenberg-Marquardt training.
abstract: The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.
authors.affiliation: Department of Electrical and Computer Engineering, Auburn University, Auburn, AL 36849-5201, USA. wilam@ieee.org
authors.forename: Bogdan M
authors.initials: BM
authors.lastname: Wilamowski
authors.forename: Hao
authors.initials: H
authors.lastname: Yu
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D006801:Humans; D007858:Learning; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D013997:Time Factors