Two highly efficient second-order algorithms for training feedforward networks.
We present two highly efficient second-order algorithms for the training of multilayer feedforward neural networks. The algorithms are based on iterations of the form employed in the Levenberg-Marquardt (LM) method for nonlinear least squares problems with the inclusion of an additional adaptive momentum term arising from the formulation of the training task as a constrained optimization problem. Their implementation requires minimal additional computations compared to a standard LM iteration. Simulations of large scale classical neural-network benchmarks are presented which reveal the power of the two methods to obtain solutions in difficult problems, whereas other standard second-order techniques (including LM) fail to converge.
Ampazis
N
N
Inst. of Informatics and Telecommun., Nat. Center for Sci. Res. "DEMOKRITOS", Athens, Greece.
Perantonis
S J
SJ