Practical selection of SVM parameters and noise estimation for SVM regression.
We investigate practical selection of hyper-parameters for support vector machines (SVM) regression (that is, epsilon-insensitive zone and regularization parameter C). The proposed methodology advocates analytic parameter selection directly from the training data, rather than re-sampling approaches commonly used in SVM applications. In particular, we describe a new analytical prescription for setting the value of insensitive zone epsilon, as a function of training sample size. Good generalization performance of the proposed parameter selection is demonstrated empirically using several low- and high-dimensional regression problems. Further, we point out the importance of Vapnik's epsilon-insensitive loss for regression problems with finite samples. To this end, we compare generalization performance of SVM regression (using proposed selection of epsilon-values) with regression using 'least-modulus' loss (epsilon=0) and standard squared loss. These comparisons indicate superior generalization performance of SVM regression under sparse sample settings, for various types of additive noise.
Cherkassky
Vladimir
V
Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, USA. cherkass@ece.umn.edu
Ma
Yunqian
Y
D001499:Bayes Theorem; D008962:Models, Theoretical; D017711:Nonlinear Dynamics; D016011:Normal Distribution; D012044:Regression Analysis; D015203:Reproducibility of Results