Entropy-based kernel mixture modeling for topographic map formation.
A new information-theoretic learning algorithm for kernel-based topographic map formation is introduced. In the one-dimensional case, the algorithm is aimed at uniformizing the cumulative distribution of the kernel mixture densities by maximizing its differential entropy. A nonparametric differential entropy estimator is used on which normalized gradient ascent is performed. Both differentiable and nondifferentiable kernels are in principle supported, such as Gaussian and rectangular (on/off) kernels. The relation is shown with joint entropy maximization of the kernel outputs. The learning algorithm's performance is assessed and compared with the theoretically optimal performance. A fixed-point rule is derived for the case of heterogeneous kernel mixtures. Finally, an extension of the algorithm to the multidimensional case is suggested.
Van Hulle
Marc M
MM
Laboratorium voor Neuro- en Psychofysiologie, Katholieke Universiteit, Campus Gasthuisberg, B-3000 Leuven, Belgium. marc@neuro.kuleuven.ac.be
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003661:Decision Support Techniques; D019277:Entropy; D016247:Information Storage and Retrieval / Q000379:methods; D007257:Information Theory; D015233:Models, Statistical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D011337:Probability Learning