title: Task-specific feature extraction and classification of fMRI volumes using a deep neural network initialized with a deep belief network: Evaluation using sensorimotor tasks.
abstract: Feedforward deep neural networks (DNNs), artificial neural networks with multiple hidden layers, have recently demonstrated a record-breaking performance in multiple areas of applications in computer vision and speech processing. Following the success, DNNs have been applied to neuroimaging modalities including functional/structural magnetic resonance imaging (MRI) and positron-emission tomography data. However, no study has explicitly applied DNNs to 3D whole-brain fMRI volumes and thereby extracted hidden volumetric representations of fMRI that are discriminative for a task performed as the fMRI volume was acquired. Our study applied fully connected feedforward DNN to fMRI volumes collected in four sensorimotor tasks (i.e., left-hand clenching, right-hand clenching, auditory attention, and visual stimulus) undertaken by 12 healthy participants. Using a leave-one-subject-out cross-validation scheme, a restricted Boltzmann machine-based deep belief network was pretrained and used to initialize weights of the DNN. The pretrained DNN was fine-tuned while systematically controlling weight-sparsity levels across hidden layers. Optimal weight-sparsity levels were determined from a minimum validation error rate of fMRI volume classification. Minimum error rates (mean±standard deviation; %) of 6.9 (±3.8) were obtained from the three-layer DNN with the sparsest condition of weights across the three hidden layers. These error rates were even lower than the error rates from the single-layer network (9.4±4.6) and the two-layer network (7.4±4.1). The estimated DNN weights showed spatial patterns that are remarkably task-specific, particularly in the higher layers. The output values of the third hidden layer represented distinct patterns/codes of the 3D whole-brain fMRI volume and encoded the information of the tasks as evaluated from representational similarity analysis. Our reported findings show the ability of the DNN to classify a single fMRI volume based on the extraction of hidden representations of fMRI volumes associated with tasks across multiple hidden layers. Our study may be beneficial to the automatic classification/diagnosis of neuropsychiatric and neurological diseases and prediction of disease severity and recovery in (pre-) clinical settings using fMRI volumes without requiring an estimation of activation patterns or ad hoc statistical evaluation.
authors.affiliation: Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea.
authors.forename: Hojin
authors.initials: H
authors.lastname: Jang
authors.affiliation: The Mind Research Network & LBERI, Albuquerque, NM, USA.
authors.forename: Sergey M
authors.initials: SM
authors.lastname: Plis
authors.affiliation: The Mind Research Network & LBERI, Albuquerque, NM, USA; Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM, USA. Electronic address: vcalhoun@mrn.org.
authors.forename: Vince D
authors.initials: VD
authors.lastname: Calhoun
authors.affiliation: Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea. Electronic address: jonghwan_lee@korea.ac.kr.
authors.forename: Jong-Hwan
authors.initials: JH
authors.lastname: Lee
mesh_terms: D000328:Adult; D001921:Brain / Q000000981:diagnostic imaging* / Q000502:physiology*; D059907:Functional Neuroimaging / Q000379:methods*; D006801:Humans; D008279:Magnetic Resonance Imaging / Q000379:methods*; D009043:Motor Activity / Q000502:physiology; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated / Q000379:methods*; D010465:Perception / Q000502:physiology