title: Direct Kernel Perceptron (DKP): ultra-fast kernel ELM-based classification with non-iterative closed-form weight calculation.
abstract: The Direct Kernel Perceptron (DKP) (Fernández-Delgado et al., 2010) is a very simple and fast kernel-based classifier, related to the Support Vector Machine (SVM) and to the Extreme Learning Machine (ELM) (Huang, Wang, & Lan, 2011), whose α-coefficients are calculated directly, without any iterative training, using an analytical closed-form expression which involves only the training patterns. The DKP, which is inspired by the Direct Parallel Perceptron, (Auer et al., 2008), uses a Gaussian kernel and a linear classifier (perceptron). The weight vector of this classifier in the feature space minimizes an error measure which combines the training error and the hyperplane margin, without any tunable regularization parameter. This weight vector can be translated, using a variable change, to the α-coefficients, and both are determined without iterative calculations. We calculate solutions using several error functions, achieving the best trade-off between accuracy and efficiency with the linear function. These solutions for the α coefficients can be considered alternatives to the ELM with a new physical meaning in terms of error and margin: in fact, the linear and quadratic DKP are special cases of the two-class ELM when the regularization parameter C takes the values C=0 and C=∞. The linear DKP is extremely efficient and much faster (over a vast collection of 42 benchmark and real-life data sets) than 12 very popular and accurate classifiers including SVM, Multi-Layer Perceptron, Adaboost, Random Forest and Bagging of RPART decision trees, Linear Discriminant Analysis, K-Nearest Neighbors, ELM, Probabilistic Neural Networks, Radial Basis Function neural networks and Generalized ART. Besides, despite its simplicity and extreme efficiency, DKP achieves higher accuracies than 7 out of 12 classifiers, exhibiting small differences with respect to the best ones (SVM, ELM, Adaboost and Random Forest), which are much slower. Thus, the DKP provides an easy and fast way to achieve classification accuracies which are not too far from the best one for a given problem. The C and Matlab code of DKP are freely available.
authors.affiliation: Centro de Investigación en Tecnoloxías da Información da USC (CITIUS), University of Santiago de Compostela, 15782, A Coruña, Spain. Electronic address: manuel.fernandez.delgado@usc.es.
authors.forename: Manuel
authors.initials: M
authors.lastname: Fernández-Delgado
authors.affiliation: Centro de Investigación en Tecnoloxías da Información da USC (CITIUS), University of Santiago de Compostela, 15782, A Coruña, Spain.
authors.forename: Eva
authors.initials: E
authors.lastname: Cernadas
authors.affiliation: Centro de Investigación en Tecnoloxías da Información da USC (CITIUS), University of Santiago de Compostela, 15782, A Coruña, Spain.
authors.forename: Senén
authors.initials: S
authors.lastname: Barro
authors.affiliation: School of Technology and Management, Viana do Castelo Polytechnic Institute, Viana do Castelo, Portugal. Electronic address: jribeiro@estg.ipvc.pt.
authors.forename: Jorge
authors.initials: J
authors.lastname: Ribeiro
authors.affiliation: DI-CCTC/Department of Informatics, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal. Electronic address: jneves@di.uminho.pt.
authors.forename: José
authors.initials: J
authors.lastname: Neves
mesh_terms: D000465:Algorithms; D002965:Classification / Q000379:methods*; D003198:Computer Simulation; D016002:Discriminant Analysis; D006801:Humans; D016014:Linear Models; D016571:Neural Networks, Computer; D060388:Support Vector Machine
keywords: Analytical weight calculation; Closed-form solution; Extreme learning machine; Kernel-based classification; Margin maximization; Parallel Delta rule; Support vector machine