Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion.
We propose an eigenvector-based heteroscedastic linear dimension reduction (LDR) technique for multiclass data. The technique is based on a heteroscedastic two-class technique which utilizes the so-called Chernoff criterion, and successfully extends the well-known linear discriminant analysis (LDA). The latter, which is based on the Fisher criterion, is incapable of dealing with heteroscedastic data in a proper way. For the two-class case, the between-class scatter is generalized so to capture differences in (co)variances. It is shown that the classical notion of between-class scatter can be associated with Euclidean distances between class means. From this viewpoint, the between-class scatter is generalized by employing the Chernoff distance measure, leading to our proposed heteroscedastic measure. Finally, using the results from the two-class case, a multiclass extension of the Chernoff criterion is proposed. This criterion combines separation information present in the class mean as well as the class covariance matrices. Extensive experiments and a comparison with similar dimension reduction techniques are presented.
Loog
Marco
M
Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands. marco@isi.uu.nl
Duin
Robert P W
RP
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D016247:Information Storage and Retrieval / Q000379:methods*; D016014:Linear Models; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity