title: Analysis of sampling techniques for imbalanced data: An n = 648 ADNI study.
abstract: Many neuroimaging applications deal with imbalanced imaging data. For example, in Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, the mild cognitive impairment (MCI) cases eligible for the study are nearly two times the Alzheimer's disease (AD) patients for structural magnetic resonance imaging (MRI) modality and six times the control cases for proteomics modality. Constructing an accurate classifier from imbalanced data is a challenging task. Traditional classifiers that aim to maximize the overall prediction accuracy tend to classify all data into the majority class. In this paper, we study an ensemble system of feature selection and data sampling for the class imbalance problem. We systematically analyze various sampling techniques by examining the efficacy of different rates and types of undersampling, oversampling, and a combination of over and undersampling approaches. We thoroughly examine six widely used feature selection algorithms to identify significant biomarkers and thereby reduce the complexity of the data. The efficacy of the ensemble techniques is evaluated using two different classifiers including Random Forest and Support Vector Machines based on classification accuracy, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity measures. Our extensive experimental results show that for various problem settings in ADNI, (1) a balanced training set obtained with K-Medoids technique based undersampling gives the best overall performance among different data sampling techniques and no sampling approach; and (2) sparse logistic regression with stability selection achieves competitive performance among various feature selection algorithms. Comprehensive experiments with various settings show that our proposed ensemble model of multiple undersampled datasets yields stable and promising results.
authors.affiliation: School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA; Center for Evolutionary Medicine and Informatics, The Biodesign Institute, Arizona State University, Tempe, AZ, USA.
authors.forename: Rashmi
authors.initials: R
authors.lastname: Dubey
authors.affiliation: School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA; Center for Evolutionary Medicine and Informatics, The Biodesign Institute, Arizona State University, Tempe, AZ, USA.
authors.forename: Jiayu
authors.initials: J
authors.lastname: Zhou
authors.affiliation: School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA.
authors.forename: Yalin
authors.initials: Y
authors.lastname: Wang
authors.affiliation: Imaging Genetics Center, Laboratory of Neuro Imaging, UCLA School of Medicine, Los Angeles, CA, USA.
authors.forename: Paul M
authors.initials: PM
authors.lastname: Thompson
authors.affiliation: School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA; Center for Evolutionary Medicine and Informatics, The Biodesign Institute, Arizona State University, Tempe, AZ, USA. Electronic address: jieping.ye@asu.edu.
authors.forename: Jieping
authors.initials: J
authors.lastname: Ye
mesh_terms: D000368:Aged; D000544:Alzheimer Disease / Q000378:metabolism / Q000473:pathology; D019540:Area Under Curve; D015415:Biomarkers / Q000032:analysis*; D001921:Brain / Q000473:pathology; D060825:Cognitive Dysfunction / Q000378:metabolism / Q000473:pathology; D005260:Female; D006801:Humans; D008297:Male; D012372:ROC Curve; D012107:Research Design; D060388:Support Vector Machine
chemical_list: D015415:Biomarkers
keywords: Alzheimer's disease; Classification; Feature selection; Imbalanced data; Oversampling; Undersampling