title: A numerical study on learning curves in stochastic multilayer feedforward networks.
abstract: The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t2 scaling is observed. This effect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from strong overfitting to effective learning.
authors.affiliation: Department of Mathematical Engineering and Inf. Physics, University of Tokyo, Japan.
authors.forename: K R
authors.initials: KR
authors.lastname: Müller
authors.forename: M
authors.initials: M
authors.lastname: Finke
authors.forename: N
authors.initials: N
authors.lastname: Murata
authors.forename: K
authors.initials: K
authors.lastname: Schulten
authors.forename: S
authors.initials: S
authors.lastname: Amari
mesh_terms: D007858:Learning / Q000502:physiology*; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*