Regularization Parameter Selections via Generalized Information Criterion.
We apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators. This approach relies heavily on the choice of regularization parameter, which controls the model complexity. In this paper, we propose employing the generalized information criterion (GIC), encompassing the commonly used Akaike information criterion (AIC) and Bayesian information criterion (BIC), for selecting the regularization parameter. Our proposal makes a connection between the classical variable selection criteria and the regularization parameter selections for the nonconcave penalized likelihood approaches. We show that the BIC-type selector enables identification of the true model consistently, and the resulting estimator possesses the oracle property in the terminology of Fan and Li (2001). In contrast, however, the AIC-type selector tends to overfit with positive probability. We further show that the AIC-type selector is asymptotically loss efficient, while the BIC-type selector is not. Our simulation results confirm these theoretical findings, and an empirical example is presented. Some technical proofs are given in the online supplementary material.
Zhang
Yiyun
Y
Yiyun Zhang is a Senior Statistician, Novartis Pharmaceuticals Corporation ( yiyun.zhang@novartis.com ). Runze Li is the correspondence author and Professor, Department of Statistics and The Methodology Center, The Pennsylvania State University, University Park, PA 16802-2111 ( rli@stat.psu.edu ). Chih-Ling Tsai is Robert W. Glock Chair professor, Graduate School of Management, University of California, Davis, CA, 95616-8609 ( cltsai@ucdavis.edu ).
Li
Runze
R
Tsai
Chih-Ling
CL