title: Feature Augmentation via Nonparametrics and Selection (FANS) in High-Dimensional Classification.
abstract: We propose a high dimensional classification method that involves nonparametric feature augmentation. Knowing that marginal density ratios are the most powerful univariate classifiers, we use the ratio estimates to transform the original feature measurements. Subsequently, penalized logistic regression is invoked, taking as input the newly transformed or augmented features. This procedure trains models equipped with local complexity and global simplicity, thereby avoiding the curse of dimensionality while creating a flexible nonlinear decision boundary. The resulting method is called Feature Augmentation via Nonparametrics and Selection (FANS). We motivate FANS by generalizing the Naive Bayes model, writing the log ratio of joint densities as a linear combination of those of marginal densities. It is related to generalized additive models, but has better interpretability and computability. Risk bounds are developed for FANS. In numerical analysis, FANS is compared with competing methods, so as to provide a guideline on its best application domain. Real data analysis demonstrates that FANS performs very competitively on benchmark email spam and gene expression data sets. Moreover, FANS is implemented by an extremely fast algorithm through parallel computing.
authors.affiliation: Jianqing Fan is Frederick L. Moore Professor of Finance, Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ, 08544 ( jqfan@princeton.edu ).
authors.forename: Jianqing
authors.initials: J
authors.lastname: Fan
authors.affiliation: Yang Feng is Assistant Professor, Department of Statistics, Columbia University, New York, NY, 10027 ( yangfeng@stat.columbia.edu ).
authors.forename: Yang
authors.initials: Y
authors.lastname: Feng
authors.affiliation: Jiancheng Jiang is Associate Professor, Department of Mathematics and Statistics, University of North Carolina at Charlotte, Charlotte, NC, 28223 ( jjiang1@uncc.edu ).
authors.forename: Jiancheng
authors.initials: J
authors.lastname: Jiang
authors.affiliation: Xin Tong is Assistant Professor, Department of Data Sciences and Operations, University of Southern California, Los Angeles, CA, 90089 ( xint@marshall.usc.edu ).
authors.forename: Xin
authors.initials: X
authors.lastname: Tong
keywords: classification; density estimation; feature augmentation; feature selection; high dimensional space; nonlinear decision boundary; parallel computing