title: Two highly efficient second-order algorithms for training feedforward networks.
abstract: We present two highly efficient second-order algorithms for the training of multilayer feedforward neural networks. The algorithms are based on iterations of the form employed in the Levenberg-Marquardt (LM) method for nonlinear least squares problems with the inclusion of an additional adaptive momentum term arising from the formulation of the training task as a constrained optimization problem. Their implementation requires minimal additional computations compared to a standard LM iteration. Simulations of large scale classical neural-network benchmarks are presented which reveal the power of the two methods to obtain solutions in difficult problems, whereas other standard second-order techniques (including LM) fail to converge.
authors.affiliation: Inst. of Informatics and Telecommun., Nat. Center for Sci. Res. "DEMOKRITOS", Athens, Greece.
authors.forename: N
authors.initials: N
authors.lastname: Ampazis
authors.forename: S J
authors.initials: SJ
authors.lastname: Perantonis