title: On robust regression with high-dimensional predictors.
abstract: We study regression M-estimates in the setting where p, the number of covariates, and n, the number of observations, are both large, but p ≤ n. We find an exact stochastic representation for the distribution of β = argmin(β∈ℝ(p)) Σ(i=1)(n) ρ(Y(i) - X(i')β) at fixed p and n under various assumptions on the objective function ρ and our statistical model. A scalar random variable whose deterministic limit rρ(κ) can be studied when p/n → κ > 0 plays a central role in this representation. We discover a nonlinear system of two deterministic equations that characterizes rρ(κ). Interestingly, the system shows that rρ(κ) depends on ρ through proximal mappings of ρ as well as various aspects of the statistical model underlying our study. Several surprising results emerge. In particular, we show that, when p/n is large enough, least squares becomes preferable to least absolute deviations for double-exponential errors.
authors: Department of Statistics, University of California, Berkeley, CA 94720, USA. nkaroui@stat.berkeley.edu
authors: Noureddine
authors: N
authors: El Karoui
authors: Derek
authors: D
authors: Bean
authors: Peter J
authors: PJ
authors: Bickel
authors: Chinghway
authors: C
authors: Lim
authors: Bin
authors: B
authors: Yu
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D016018:Least-Squares Analysis; D016014:Linear Models; D013269:Stochastic Processes
keywords: concentration of measure; high-dimensional statistics; prox function
references: Proc Natl Acad Sci U S A. 2013 Sep 3;110(36):14563-8
references: 23954907