title: Fast Gaussian kernel learning for classification tasks based on specially structured global optimization.
abstract: For a practical pattern classification task solved by kernel methods, the computing time is mainly spent on kernel learning (or training). However, the current kernel learning approaches are based on local optimization techniques, and hard to have good time performances, especially for large datasets. Thus the existing algorithms cannot be easily extended to large-scale tasks. In this paper, we present a fast Gaussian kernel learning method by solving a specially structured global optimization (SSGO) problem. We optimize the Gaussian kernel function by using the formulated kernel target alignment criterion, which is a difference of increasing (d.i.) functions. Through using a power-transformation based convexification method, the objective criterion can be represented as a difference of convex (d.c.) functions with a fixed power-transformation parameter. And the objective programming problem can then be converted to a SSGO problem: globally minimizing a concave function over a convex set. The SSGO problem is classical and has good solvability. Thus, to find the global optimal solution efficiently, we can adopt the improved Hoffman's outer approximation method, which need not repeat the searching procedure with different starting points to locate the best local minimum. Also, the proposed method can be proven to converge to the global solution for any classification task. We evaluate the proposed method on twenty benchmark datasets, and compare it with four other Gaussian kernel learning methods. Experimental results show that the proposed method stably achieves both good time-efficiency performance and good classification performance.
authors.affiliation: College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108, China; Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing (Fuzhou University), Fuzhou 350108, China. Electronic address: spzhong@fzu.edu.cn.
authors.forename: Shangping
authors.initials: S
authors.lastname: Zhong
authors.affiliation: College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108, China; Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing (Fuzhou University), Fuzhou 350108, China.
authors.forename: Tianshun
authors.initials: T
authors.lastname: Chen
authors.affiliation: College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108, China; Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing (Fuzhou University), Fuzhou 350108, China.
authors.forename: Fengying
authors.initials: F
authors.lastname: He
authors.affiliation: College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108, China; Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing (Fuzhou University), Fuzhou 350108, China.
authors.forename: Yuzhen
authors.initials: Y
authors.lastname: Niu
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D002965:Classification / Q000379:methods*; D016011:Normal Distribution; D012984:Software
keywords: Difference of convex functions; Difference of increasing functions; Fast Gaussian kernel learning method; Hoffman’s outer approximation method; Kernel target alignment criterion; Specially structured global optimization