Generative models for discovering sparse distributed representations.
We describe a hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demonstrate that the network learns to extract sparse, distributed, hierarchical representations.
Hinton
G E
GE
Department of Computer Science, University of Toronto, Ontario, Canada.
Ghahramani
Z
Z
D000465:Algorithms; D002540:Cerebral Cortex / Q000502:physiology; D006801:Humans; D016015:Logistic Models; D016571:Neural Networks, Computer; D016011:Normal Distribution; D010465:Perception; D012890:Sleep; D014851:Wakefulness