Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor autodetermination.
The nature of Bayesian Ying-Yang harmony learning is reexamined from an information theoretic perspective. Not only its ability for model selection and regularization is explained with new insights, but also discussions are made on its relations and differences from the studies of minimum description length (MDL), Bayesian approach, the bit-back based MDL, Akaike information criterion (AIC), maximum likelihood, information geometry, Helmholtz machines, and variational approximation. Moreover, a generalized projection geometry is introduced for further understanding such a new mechanism. Furthermore, new algorithms are also developed for implementing Gaussian factor analysis (FA) and non-Gaussian factor analysis (NFA) such that selecting appropriate factors is automatically made during parameter learning.
Xu
Lei
L
Department of Computer Science and Engineering, Chinese University of Hong Kong, Shatin, NT, Hong Kong, PR China. lxu@cse.cuhk.edu.hk
D000465:Algorithms; D001185:Artificial Intelligence; D001499:Bayes Theorem; D003198:Computer Simulation; D003661:Decision Support Techniques; D005163:Factor Analysis, Statistical; D016247:Information Storage and Retrieval / Q000379:methods*; D007257:Information Theory; D015233:Models, Statistical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D011337:Probability Learning