The Rosenblatt Bayesian algorithm learning in a nonstationary environment.
In this letter, we study online learning in neural networks (NNs) obtained by approximating Bayesian learning. The approach is applied to Gibbs learning with the Rosenblatt potential in a nonstationary environment. The online scheme is obtained by the minimization (maximization) of the Kullback-Leibler divergence (cross entropy) between the true posterior distribution and the parameterized one. The complexity of the learning algorithm is further decreased by projecting the posterior onto a Gaussian distribution and imposing a spherical covariance matrix. We study in detail the particular case of learning linearly separable rules. In the case of a fixed rule, we observe an asymptotic generalization error e(g) infinity alpha(-1) for both the spherical and the full covariance matrix approximations. However, in the case of drifting rule, only the full covariance matrix algorithm shows a good performance. This good performance is indeed a surprise since the algorithm is obtained by projecting without the benefit of the extra information on drifting.
de Oliveira
Evaldo Araújo
EA
D000465:Algorithms; D001185:Artificial Intelligence; D001499:Bayes Theorem; D003198:Computer Simulation; D005246:Feedback; D016247:Information Storage and Retrieval / Q000379:methods*; D008962:Models, Theoretical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated / Q000379:methods*; D013269:Stochastic Processes