title: A Bayesian approach to joint feature selection and classifier design.
abstract: This paper adopts a Bayesian approach to simultaneously learn both an optimal nonlinear classifier and a subset of predictor variables (or features) that are most relevant to the classification task. The approach uses heavy-tailed priors to promote sparsity in the utilization of both basis functions and features; these priors act as regularizers for the likelihood function that rewards good classification on the training data. We derive an expectation-maximization (EM) algorithm to efficiently compute a maximum a posteriori (MAP) point estimate of the various parameters. The algorithm is an extension of recent state-of-the-art sparse Bayesian classifiers, which in turn can be seen as Bayesian counterparts of support vector machines. Experimental comparisons using kernel classifiers demonstrate both parsimonious feature selection and excellent classification accuracy on a range of synthetic and benchmark data sets.
authors.affiliation: Department of Electrical Engineering, Duke University, Durham, NC 27708-0291, USA. balaji@ee.duke.edu
authors.forename: Balaji
authors.initials: B
authors.lastname: Krishnapuram
authors.forename: Alexander J
authors.initials: AJ
authors.lastname: Hartemink
authors.forename: Lawrence
authors.initials: L
authors.lastname: Carin
authors.forename: Mário A T
authors.initials: MA
authors.lastname: Figueiredo
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001499:Bayes Theorem; D014408:Biomarkers, Tumor / Q000235:genetics; D016000:Cluster Analysis; D003110:Colonic Neoplasms / Q000175:diagnosis / Q000235:genetics; D003198:Computer Simulation; D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D020869:Gene Expression Profiling / Q000379:methods*; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods; D007938:Leukemia / Q000175:diagnosis / Q000235:genetics; D008954:Models, Biological; D015233:Models, Statistical; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
chemical_list: D014408:Biomarkers, Tumor