title: Framework for making better predictions by directly estimating variables' predictivity.
abstract: We propose approaching prediction from a framework grounded in the theoretical correct prediction rate of a variable set as a parameter of interest. This framework allows us to define a measure of predictivity that enables assessing variable sets for, preferably high, predictivity. We first define the prediction rate for a variable set and consider, and ultimately reject, the naive estimator, a statistic based on the observed sample data, due to its inflated bias for moderate sample size and its sensitivity to noisy useless variables. We demonstrate that the [Formula: see text]-score of the PR method of VS yields a relatively unbiased estimate of a parameter that is not sensitive to noisy variables and is a lower bound to the parameter of interest. Thus, the PR method using the [Formula: see text]-score provides an effective approach to selecting highly predictive variables. We offer simulations and an application of the [Formula: see text]-score on real data to demonstrate the statistic's predictive performance on sample data. We conjecture that using the partition retention and [Formula: see text]-score can aid in finding variable sets with promising prediction rates; however, further research in the avenue of sample-based measures of predictivity is much desired.
authors.affiliation: Department of Politics, Princeton University, Princeton, NJ 08540.
authors.forename: Adeline
authors.identifier: 0000-0001-5791-5541
authors.initials: A
authors.lastname: Lo
authors.affiliation: Department of Statistics, Harvard University, Cambridge, MA 02138; slo@stat.columbia.edu chernoff@stat.harvard.edu tz33@columbia.edu.
authors.forename: Herman
authors.initials: H
authors.lastname: Chernoff
authors.affiliation: Department of Statistics, Columbia University, New York, NY 10027 slo@stat.columbia.edu chernoff@stat.harvard.edu tz33@columbia.edu.
authors.forename: Tian
authors.identifier: 0000-0003-4889-0391
authors.initials: T
authors.lastname: Zheng
authors.affiliation: Department of Statistics, Columbia University, New York, NY 10027 slo@stat.columbia.edu chernoff@stat.harvard.edu tz33@columbia.edu.
authors.forename: Shaw-Hwa
authors.initials: SH
authors.lastname: Lo
keywords: high-dimensional data; prediction; predictivity; variable selection