A systematic comparison of supervised classifiers.
Pattern recognition has been employed in a myriad of industrial, commercial and academic applications. Many techniques have been devised to tackle such a diversity of applications. Despite the long tradition of pattern recognition research, there is no technique that yields the best classification in all scenarios. Therefore, as many techniques as possible should be considered in high accuracy applications. Typical related works either focus on the performance of a given algorithm or compare various classification methods. In many occasions, however, researchers who are not experts in the field of machine learning have to deal with practical classification tasks without an in-depth knowledge about the underlying parameters. Actually, the adequate choice of classifiers and parameters in such practical circumstances constitutes a long-standing problem and is one of the subjects of the current paper. We carried out a performance study of nine well-known classifiers implemented in the Weka framework and compared the influence of the parameter configurations on the accuracy. The default configuration of parameters in Weka was found to provide near optimal performance for most cases, not including methods such as the support vector machine (SVM). In addition, the k-nearest neighbor method frequently allowed the best accuracy. In certain conditions, it was possible to improve the quality of SVM by more than 20% with respect to their default parameter configuration.
Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, São Paulo, Brazil.
Diego Raphael
DR
Amancio
São Carlos Institute of Physics, University of São Paulo, São Carlos, São Paulo, Brazil.
Cesar Henrique
CH
Comin
São Carlos Institute of Physics, University of São Paulo, São Carlos, São Paulo, Brazil.
Dalcimar
D
Casanova
São Carlos Institute of Physics, University of São Paulo, São Carlos, São Paulo, Brazil.
Gonzalo
G
Travieso
São Carlos Institute of Physics, University of São Paulo, São Carlos, São Paulo, Brazil.
Odemir Martinez
OM
Bruno
Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, São Paulo, Brazil.
Francisco Aparecido
FA
Rodrigues
São Carlos Institute of Physics, University of São Paulo, São Carlos, São Paulo, Brazil.
Luciano da Fontoura
Lda F
Costa
D003627:Data Interpretation, Statistical; D012372:ROC Curve; D060388:Support Vector Machine