On constrained and regularized high-dimensional regression.
High-dimensional feature selection has become increasingly crucial for seeking parsimonious models in estimation. For selection consistency, we derive one necessary and sufficient condition formulated on the notion of degree-of-separation. The minimal degree of separation is necessary for any method to be selection consistent. At a level slightly higher than the minimal degree of separation, selection consistency is achieved by a constrainedL0-method and its computational surrogate-the constrained truncatedL1-method. This permits up to exponentially many features in the sample size. In other words, these methods are optimal in feature selection against any selection method. In contrast, their regularization counterparts-theL0-regularization and truncatedL1-regularization methods enable so under slightly stronger assumptions. More importantly, sharper parameter estimation/prediction is realized through such selection, leading to minimax parameter estimation. This, otherwise, is impossible in absence of a good selection method for high-dimensional analysis.
School of Statistics, University of Minnesota, Minneapolis, MN 55455.
Xiaotong
X
Shen
Division of Biostatistics, University of Minnesota, Minneapolis, MN 55455.
Wei
W
Pan
School of Statistics, University of Minnesota, Minneapolis, MN 55455.
Yunzhang
Y
Zhu
Division of Biostatistics, University of Minnesota, Minneapolis, MN 55455.
Hui
H
Zhou
(p, n) versus fixed p-asymptotics; Constrained regression; difference convex programming; nonconvex regularization; parameter and nonparametric models