title: A Generic Path Algorithm for Regularized Statistical Estimation.
abstract: Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution towards prior information. In general, a regularized estimation problem minimizes the sum of a loss function and a penalty term. The penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized ℓ1penalized regression methods. In this article we follow a recent idea by Wu (2011, 2012) and propose an exact path solver based on ordinary differential equations (EPSODE) that works for any convex loss function and can deal with generalized ℓ1penalties as well as more complicated regularization such as inequality constraints encountered in shape-restricted regressions and nonparametric density estimation. Non-asymptotic error bounds for the equality regularized estimates are derived. In practice, the EPSODE can be coupled with AIC, BIC,Cpor cross-validation to select an optimal tuning parameter, or provides a convenient model space for performing model averaging or aggregation. Our applications to generalized ℓ1regularized generalized linear models, shape-restricted regressions, Gaussian graphical models, and nonparametric density estimation showcase the potential of the EPSODE algorithm.
authors.affiliation: Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203.
authors.forename: Hua
authors.initials: H
authors.lastname: Zhou
authors.affiliation: Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203.
authors.forename: Yichao
authors.initials: Y
authors.lastname: Wu
keywords: Gaussian graphical model; generalized linear model; lasso; log-concave density estimation; ordinary differential equations; quasi-likelihoods; regularization; shape restricted regression; solution path