Ideal regularization for learning kernels from labels.
In this paper, we propose a new form of regularization that is able to utilize the label information of a data set for learning kernels. The proposed regularization, referred to as ideal regularization, is a linear function of the kernel matrix to be learned. The ideal regularization allows us to develop efficient algorithms to exploit labels. Three applications of the ideal regularization are considered. Firstly, we use the ideal regularization to incorporate the labels into a standard kernel, making the resulting kernel more appropriate for learning tasks. Next, we employ the ideal regularization to learn a data-dependent kernel matrix from an initial kernel matrix (which contains prior similarity information, geometric structures, and labels of the data). Finally, we incorporate the ideal regularization to some state-of-the-art kernel learning problems. With this regularization, these learning problems can be formulated as simpler ones which permit more efficient solvers. Empirical results show that the ideal regularization exploits the labels effectively and efficiently.
College of Mathematics and Computational Science, Shenzhen University, Shenzhen, China; School of Mathematics and Computational Science, Sun Yat-sen University, Guangzhou, China. Electronic address: pbb@szu.edu.cn.
Binbin
B
Pan
School of Information Science and Technology, Sun Yat-sen University, Guangzhou, China. Electronic address: stsljh@mail.sysu.edu.cn.
Jianhuang
J
Lai
Department of Mathematics, Syracuse University, Syracuse, NY 13244, USA. Electronic address: lshen03@syr.edu.
Lixin
L
Shen
D000465:Algorithms; D001185:Artificial Intelligence; D016208:Databases, Factual; D006801:Humans; D016014:Linear Models; D016011:Normal Distribution; D013995:Time
Ideal kernel; Kernel methods; Labels; Regularization; Semi-supervised learning; von Neumann divergence