Cross-entropy embedding of high-dimensional data using the neural gas model.
A cross-entropy approach to mapping high-dimensional data into a low-dimensional space embedding is presented. The method allows to project simultaneously the input data and the codebook vectors, obtained with the Neural Gas (NG) quantizer algorithm, into a low-dimensional output space. The aim of this approach is to preserve the relationship defined by the NG neighborhood function for each pair of input and codebook vectors. A cost function based on the cross-entropy between input and output probabilities is minimized by using a Newton-Raphson method. The new approach is compared with Sammon's non-linear mapping (NLM) and the hierarchical approach of combining a vector quantizer such as the self-organizing feature map (SOM) or NG with the NLM recall algorithm. In comparison with these techniques, our method delivers a clear visualization of both data points and codebooks, and it achieves a better mapping quality in terms of the topology preservation measure q(m).
Estévez
Pablo A
PA
Department of Electrical Engineering, University of Chile, Casilla 412-3, Santiago, Chile. pestevez@ing.uchile.cl
Figueroa
Cristián J
CJ
Saito
Kazumi
K
D000465:Algorithms; D001185:Artificial Intelligence; D003627:Data Interpretation, Statistical; D016208:Databases, Factual; D019277:Entropy; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008959:Models, Neurological; D017286:Polysomnography / Q000706:statistics & numerical data