title: Selecting Relevant Descriptors for Classification by Bayesian Estimates: A Comparison with Decision Trees and Support Vector Machines Approaches for Disparate Data Sets.
abstract: Classification algorithms suffer from the curse of dimensionality, which leads to overfitting, particularly if the problem is over-determined. Therefore it is of particular interest to identify the most relevant descriptors to reduce the complexity. We applied Bayesian estimates to model the probability distribution of descriptors values used for binary classification using n-fold cross-validation. As a measure for the discriminative power of the classifiers, the symmetric form of the Kullback-Leibler divergence of their probability distributions was computed. We found that the most relevant descriptors possess a Gaussian-like distribution of their values, show the largest divergences, and therefore appear most often in the cross-validation scenario. The results were compared to those of the LASSO feature selection method applied to multiple decision trees and support vector machine approaches for data sets of substrates and nonsubstrates of three Cytochrome P450 isoenzymes, which comprise strongly unbalanced compound distributions. In contrast to decision trees and support vector machines, the performance of Bayesian estimates is less affected by unbalanced data sets. This strategy reveals those descriptors that allow a simple linear separation of the classes, whereas the superior accuracy of decision trees and support vector machines can be attributed to nonlinear separation, which are in turn more prone to overfitting.
authors.affiliation: Section of Biostatistics, Paul-Ehrlich-Institut, Federal Institute for Vaccines and Biomedicines, Paul-Ehrlich-Straße 51-59, 63225 Langen, Germany.
authors.forename: Miriam
authors.initials: M
authors.lastname: Carbon-Mangels
authors.affiliation: Center for Bioinformatics, Saarland University, Campus Building E2.1, 66123 Saarbrücken, Germany phone/fax: +49 681 302 70703/70702. michael.hutter@bioinformatik.uni-saarland.de.
authors.forename: Michael C
authors.initials: MC
authors.lastname: Hutter
keywords: Chemoinformatics; Drug design; Feature selection; Machine learning; Molecular descriptors; Naïve Bayes classifier; in silico-ADMET