Probabilistic sequential independent components analysis.
Under-complete models, which derive lower dimensional representations of input data, are valuable in domains in which the number of input dimensions is very large, such as data consisting of a temporal sequence of images. This paper presents the under-complete product of experts (UPoE), where each expert models a one-dimensional projection of the data. Maximum-likelihood learning rules for this model constitute a tractable and exact algorithm for learning under-complete independent components. The learning rules for this model coincide with approximate learning rules proposed earlier for under-complete independent component analysis (UICA) models. This paper also derives an efficient sequential learning algorithm from this model and discusses its relationship to sequential independent component analysis (ICA), projection pursuit density estimation, and feature induction algorithms for additive random field models. This paper demonstrates the efficacy of these novel algorithms on high-dimensional continuous datasets.
Welling
Max
M
Department of Computer Science, University of Toronto, Toronto M5S 3G4 ON, Canada. welling@ics.uci.edu
Zemel
Richard S
RS
Hinton
Geoffrey E
GE
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003661:Decision Support Techniques; D005103:Expert Systems; D007090:Image Interpretation, Computer-Assisted / Q000379:methods; D016247:Information Storage and Retrieval / Q000379:methods; D007257:Information Theory; D015233:Models, Statistical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D025341:Principal Component Analysis; D011337:Probability Learning