Optimal design of regularization term and regularization parameter by subspace information criterion.
The problem of designing the regularization term and regularization parameter for linear regression models is discussed. Previously, we derived an approximation to the generalization error called the subspace information criterion (SIC), which is an unbiased estimator of the generalization error with finite samples under certain conditions. In this paper, we apply SIC to regularization learning and use it for: (a) choosing the optimal regularization term and regularization parameter from the given candidates; (b) obtaining the closed form of the optimal regularization parameter for a fixed regularization term. The effectiveness of SIC is demonstrated through computer simulations with artificial and real data.
Sugiyama
Masashi
M
Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, Japan. sugi@og.cs.titech.ac.jp
Ogawa
Hidemitsu
H
D016014:Linear Models; D008962:Models, Theoretical