Fast exact leave-one-out cross-validation of sparse least-squares support vector machines.
Leave-one-out cross-validation has been shown to give an almost unbiased estimator of the generalisation properties of statistical models, and therefore provides a sensible criterion for model selection and comparison. In this paper we show that exact leave-one-out cross-validation of sparse Least-Squares Support Vector Machines (LS-SVMs) can be implemented with a computational complexity of only O(ln2) floating point operations, rather than the O(l2n2) operations of a naïve implementation, where l is the number of training patterns and n is the number of basis vectors. As a result, leave-one-out cross-validation becomes a practical proposition for model selection in large scale applications. For clarity the exposition concentrates on sparse least-squares support vector machines in the context of non-linear regression, but is equally applicable in a pattern recognition setting.
Cawley
Gavin C
GC
School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK. gcc@cmp.uea.ac.uk
Talbot
Nicola L C
NL
D000465:Algorithms; D016018:Least-Squares Analysis; D008433:Mathematics; D015233:Models, Statistical; D016571:Neural Networks, Computer; D017711:Nonlinear Dynamics; D010363:Pattern Recognition, Automated / Q000379:methods