title: Leap-frog is a robust algorithm for training neural networks.
abstract: Optimization of perceptron neural network classifiers requires an optimization algorithm that is robust. In general, the best network is selected after a number of optimization trials. An effective optimization algorithm generates good weight-vector solutions in a few optimization trial runs owing to its inherent ability to escape local minima, where a less effective algorithm requires a larger number of trial runs. Repetitive training and testing is a tedious process, so that an effective algorithm is desirable to reduce training time and increase the quality of the set of available weight-vector solutions. We present leap-frog as a robust optimization algorithm for training neural networks. In this paper the dynamic principles of leap-frog are described together with experiments to show the ability of leap-frog to generate reliable weight-vector solutions. Performance histograms are used to compare leap-frog with a variable-metric method, a conjugate-gradient method with modified restarts, and a constrained-momentum-based algorithm. Results indicate that leap-frog performs better in terms of classification error than the remaining three algorithms on two distinctly different test problems.
authors.affiliation: Department of Electrical and Electronic Engineering, University of Pretoria, South Africa.
authors.forename: J E
authors.initials: JE
authors.lastname: Holm
authors.forename: E C
authors.initials: EC
authors.lastname: Botha
mesh_terms: D000465:Algorithms; D008962:Models, Theoretical; D016571:Neural Networks, Computer; D013663:Teaching / Q000379:methods