Implementing Gaussian process inference with neural networks.
Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.
Frean
Marcus
M
Victoria University of Wellington, P.O. Box 600, Wellington, New Zealand. marcus@mcs.vuw.ac.nz
Lilley
Matt
M
Boyle
Phillip
P
D000465:Algorithms; D001185:Artificial Intelligence; D001921:Brain / Q000502:physiology; D009415:Nerve Net / Q000502:physiology; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D016011:Normal Distribution