title: Complexity issues in natural gradient descent method for training multilayer perceptrons.
abstract: The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).
authors.affiliation: Oregon Graduate Institute, Computer Science Dept, Box 91000, Portland OR 97291, USA. hyang@cse.ogi.edu
authors.forename: H H
authors.initials: HH
authors.lastname: Yang
authors.forename: S
authors.initials: S
authors.lastname: Amari
mesh_terms: D007858:Learning; D008959:Models, Neurological; D016571:Neural Networks, Computer; D009474:Neurons; D013269:Stochastic Processes