title: Generic Feature Selection with Short Fat Data.
abstract: Consider a regression problem in which there are many more explanatory variables than data points,i.e.,p≫n. Essentially, without reducing the number of variables inference is impossible. So, we group thepexplanatory variables into blocks by clustering, evaluate statistics on the blocks and then regress the response on these statistics under a penalized error criterion to obtain estimates of the regression coefficients. We examine the performance of this approach for a variety of choices ofn,p, classes of statistics, clustering algorithms, penalty terms, and data types. Whennis not large, the discrimination over number of statistics is weak, but computations suggest regressing on approximately [n/K] statistics whereKis the number of blocks formed by a clustering algorithm. Small deviations from this are observed when the blocks of variables are of very different sizes. Larger deviations are observed when the penalty term is anLqnorm with high enoughq.
authors.affiliation: Department of Statistics, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA.
authors.forename: B
authors.initials: B
authors.lastname: Clarke
authors.affiliation: Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115.
authors.forename: J-H
authors.initials: JH
authors.lastname: Chu
keywords: Bridge; Clustering; LASSO; Large p small n; Ridge; Summary statistics; Variance-bias tradeoff