Feature selection in MLPs and SVMs based on maximum output information.
This paper presents feature selection algorithms for multilayer perceptrons (MLPs) and multiclass support vector machines (SVMs), using mutual information between class labels and classifier outputs, as an objective function. This objective function involves inexpensive computation of information measures only on discrete variables; provides immunity to prior class probabilities; and brackets the probability of error of the classifier. The maximum output information (MOI) algorithms employ this function for feature subset selection by greedy elimination and directed search. The output of the MOI algorithms is a feature subset of user-defined size and an associated trained classifier (MLP/SVM). These algorithms compare favorably with a number of other methods in terms of performance on various artificial and real-world data sets.
Sindhwani
Vikas
V
Department of Computer Science, the University of Chicago, IL 60637, USA.
Rakshit
Subrata
S
Deodhare
Dipti
D
Erdogmus
Deniz
D
Principe
Jose C
JC
Niyogi
Partha
P
D000465:Algorithms; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000145:classification / Q000175:diagnosis; D016000:Cluster Analysis; D003198:Computer Simulation; D003205:Computing Methodologies; D003661:Decision Support Techniques; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods; D007257:Information Theory; D016013:Likelihood Functions; D015233:Models, Statistical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D011337:Probability Learning