title: Implementing Gaussian process inference with neural networks.
abstract: Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.
authors.affiliation: Victoria University of Wellington, P.O. Box 600, Wellington, New Zealand. marcus@mcs.vuw.ac.nz
authors.forename: Marcus
authors.initials: M
authors.lastname: Frean
authors.forename: Matt
authors.initials: M
authors.lastname: Lilley
authors.forename: Phillip
authors.initials: P
authors.lastname: Boyle
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001921:Brain / Q000502:physiology; D009415:Nerve Net / Q000502:physiology; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D016011:Normal Distribution