Gaussian processes for classification: mean-field algorithms.
We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler "naive" mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.
Opper
M
M
Department of Computer Science and Applied Mathematics, Aston University, Birmingham, UK.
Winther
O
O
D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D001499:Bayes Theorem; D003386:Brachyura / Q000033:anatomy & histology; D002965:Classification; D003198:Computer Simulation; D003924:Diabetes Mellitus, Type 2 / Q000208:ethnology / Q000235:genetics; D004193:Discrimination Learning / Q000502:physiology; D005260:Female; D020022:Genetic Predisposition to Disease; D006801:Humans; D007198:Indians, North American / Q000235:genetics; D016013:Likelihood Functions; D008297:Male; D008959:Models, Neurological; D016011:Normal Distribution; D012727:Sex Characteristics; D013018:Sound Spectrography