title: Comparison of Feature Selection Techniques in Machine Learning for Anatomical Brain MRI in Dementia.
abstract: We present a comparative split-half resampling analysis of various data driven feature selection and classification methods for the whole brain voxel-based classification analysis of anatomical magnetic resonance images. We compared support vector machines (SVMs), with or without filter based feature selection, several embedded feature selection methods and stability selection. While comparisons of the accuracy of various classification methods have been reported previously, the variability of the out-of-training sample classification accuracy and the set of selected features due to independent training and test sets have not been previously addressed in a brain imaging context. We studied two classification problems: 1) Alzheimer's disease (AD) vs. normal control (NC) and 2) mild cognitive impairment (MCI) vs. NC classification. In AD vs. NC classification, the variability in the test accuracy due to the subject sample did not vary between different methods and exceeded the variability due to different classifiers. In MCI vs. NC classification, particularly with a large training set, embedded feature selection methods outperformed SVM-based ones with the difference in the test accuracy exceeding the test accuracy variability due to the subject sample. The filter and embedded methods produced divergent feature patterns for MCI vs. NC classification that suggests the utility of the embedded feature selection for this problem when linked with the good generalization performance. The stability of the feature sets was strongly correlated with the number of features selected, weakly correlated with the stability of classification accuracy, and uncorrelated with the average classification accuracy.
authors.affiliation: Department of Bioengineering and Aerospace Engineering, Universidad Carlos III de Madrid, Avd. de la Universidad, 30, 28911, Leganes, Spain. jtohka@ing.uc3m.es.|Instituto de Investigación Sanitaria Gregorio Marañon, Madrid, Spain. jtohka@ing.uc3m.es.
authors.forename: Jussi
authors.identifier: 0000-0002-1048-5860
authors.initials: J
authors.lastname: Tohka
authors.affiliation: Department of Signal Processing, Tampere University of Technology, P.O. Box 553, FI-33101, Tampere, Finland.
authors.forename: Elaheh
authors.initials: E
authors.lastname: Moradi
authors.affiliation: Department of Signal Processing, Tampere University of Technology, P.O. Box 553, FI-33101, Tampere, Finland.
authors.forename: Heikki
authors.initials: H
authors.lastname: Huttunen
mesh_terms: D000544:Alzheimer Disease / Q000145:classification / Q000000981:diagnostic imaging / Q000473:pathology; D001921:Brain / Q000000981:diagnostic imaging* / Q000473:pathology*; D001931:Brain Mapping / Q000379:methods*; D060825:Cognitive Dysfunction / Q000145:classification / Q000000981:diagnostic imaging / Q000473:pathology; D016208:Databases, Factual; D003704:Dementia / Q000145:classification / Q000000981:diagnostic imaging* / Q000473:pathology*; D006801:Humans; D000069550:Machine Learning; D008279:Magnetic Resonance Imaging / Q000379:methods*; D010363:Pattern Recognition, Automated; D060388:Support Vector Machine
keywords: Alzheimer’s Disease; Classification; Feature selection; Machine Learning; Magnetic Resonance Imaging; Multivariate pattern analysis