title: Parallel nonlinear optimization techniques for training neural networks.
abstract: In this paper, we propose the use of parallel quasi-Newton (QN) optimization techniques to improve the rate of convergence of the training process for neural networks. The parallel algorithms are developed by using the self-scaling quasi-Newton (SSQN) methods. At the beginning of each iteration, a set of parallel search directions is generated. Each of these directions is selectively chosen from a representative class of QN methods. Inexact line searches are then carried out to estimate the minimum point along each search direction. The proposed parallel algorithms are tested over a set of nine benchmark problems. Computational results show that the proposed algorithms outperform other existing methods, which are evaluated over the same set of test problems.
authors.affiliation: Dept. of Comput. Sci., Nat. Univ. of Singapore, Singapore.
authors.forename: P H
authors.initials: PH
authors.lastname: Phua
authors.forename: Daohua
authors.initials: D
authors.lastname: Ming