Feature selection in MLPs and SVMs based on maximum output information.
This paper presents feature selection algorithms for multilayer perceptrons (MLPs) and multiclass support vector machines (SVMs), using mutual information between class labels and classifier outputs, as an objective function. This objective function involves inexpensive computation of information measures only on discrete variables; provides immunity to prior class probabilities; and brackets the probability of error of the classifier. The maximum output information (MOI) algorithms employ this function for feature subset selection by greedy elimination and directed search. The output of the MOI algorithms is a feature subset of user-defined size and an associated trained classifier (MLP/SVM). These algorithms compare favorably with a number of other methods in terms of performance on various artificial and real-world data sets.
Department of Computer Science, the University of Chicago, IL 60637, USA.
Vikas
V
Sindhwani
Subrata
S
Rakshit
Dipti
D
Deodhare
Deniz
D
Erdogmus
Jose C
JC
Principe
Partha
P
Niyogi
D000465:Algorithms; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000145:classification / Q000175:diagnosis; D016000:Cluster Analysis; D003198:Computer Simulation; D003205:Computing Methodologies; D003661:Decision Support Techniques; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods; D007257:Information Theory; D016013:Likelihood Functions; D015233:Models, Statistical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated; D011337:Probability Learning