Incremental learning for ν-Support Vector Regression.
The ν-Support Vector Regression (ν-SVR) is an effective regression learning algorithm, which has the advantage of using a parameter ν on controlling the number of support vectors and adjusting the width of the tube automatically. However, compared to ν-Support Vector Classification (ν-SVC) (Schölkopf et al., 2000), ν-SVR introduces an additional linear term into its objective function. Thus, directly applying the accurate on-line ν-SVC algorithm (AONSVM) to ν-SVR will not generate an effective initial solution. It is the main challenge to design an incremental ν-SVR learning algorithm. To overcome this challenge, we propose a special procedure called initial adjustments in this paper. This procedure adjusts the weights of ν-SVC based on the Karush-Kuhn-Tucker (KKT) conditions to prepare an initial solution for the incremental learning. Combining the initial adjustments with the two steps of AONSVM produces an exact and effective incremental ν-SVR learning algorithm (INSVR). Theoretical analysis has proven the existence of the three key inverse matrices, which are the cornerstones of the three steps of INSVR (including the initial adjustments), respectively. The experiments on benchmark datasets demonstrate that INSVR can avoid the infeasible updating paths as far as possible, and successfully converges to the optimal solution. The results also show that INSVR is faster than batch ν-SVR algorithms with both cold and warm starts.
Jiangsu Engineering Center of Network Monitoring, Nanjing University of Information Science & Technology, Nanjing, PR China; Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, PR China; School of Computer & Software, Nanjing University of Information Science & Technology, Nanjing, PR China; Department of Medical Biophysics, University of Western Ontario, London, Ontario, Canada. Electronic address: jsgubin@nuist.edu.cn.
Bin
B
Gu
Department of Computer Science, University of Central Arkansas, Conway, AR, USA. Electronic address: ssheng@uca.edu.
Victor S
VS
Sheng
GE Health Care, London, Ontario, Canada. Electronic address: zhijie@ualberta.ca.
Zhijie
Z
Wang
Victoria Hospital, London Health Science Center, London, Ontario, Canada. Electronic address: derek.ho@lhsc.on.ca.
Derek
D
Ho
St. Joseph's Health Care, London, Ontario, Canada. Electronic address: sidosman@hotmail.com.
Said
S
Osman
GE Health Care, London, Ontario, Canada; Department of Medical Biophysics, University of Western Ontario, London, Ontario, Canada. Electronic address: Shuo.Li@ge.com.
Shuo
S
Li
D000465:Algorithms; D006801:Humans; D016014:Linear Models; D000069550:Machine Learning; D016011:Normal Distribution; D060388:Support Vector Machine
-Support Vector Regression; Incremental learning; Online learning; Support vector machine