Asymptotic analysis of Bayesian generalization error with Newton diagram.
Statistical learning machines that have singularities in the parameter space, such as hidden Markov models, Bayesian networks, and neural networks, are widely used in the field of information engineering. Singularities in the parameter space determine the accuracy of estimation in the Bayesian scenario. The Newton diagram in algebraic geometry is recognized as an effective method by which to investigate a singularity. The present paper proposes a new technique to plug the diagram in the Bayesian analysis. The proposed technique allows the generalization error to be clarified and provides a foundation for an efficient model selection. We apply the proposed technique to mixtures of binomial distributions.
Yamazaki
Keisuke
K
Precision and Intelligence Laboratory, Tokyo Institute of Technology, R2-5, 4259 Nagatsuta, Midori-ku, Yokohama 226-8503 226-8503, Japan. k-yam@pi.titech.ac.jp
Aoyagi
Miki
M
Watanabe
Sumio
S
D000465:Algorithms; D001185:Artificial Intelligence; D001499:Bayes Theorem; D003198:Computer Simulation; D001330:Electronic Data Processing; D005793:Generalization, Psychological; D006801:Humans; D016247:Information Storage and Retrieval