title: DeepNAT: Deep convolutional neural network for segmenting neuroanatomy.
abstract: We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.
authors.affiliation: Department of Child and Adolescent Psychiatry, Psychosomatic and Psychotherapy, Ludwig-Maximilian-University, Waltherstr. 23, 81369 München, Munich, Germany. Electronic address: christian.wachinger@med.uni-muenchen.de.
authors.forename: Christian
authors.initials: C
authors.lastname: Wachinger
authors.affiliation: Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA; German Centre for Neurodegenerative Diseases (DZNE), Department of Image Analysis, Bonn, Germany.
authors.forename: Martin
authors.initials: M
authors.lastname: Reuter
authors.affiliation: SAP SE, Berlin, Germany.
authors.forename: Tassilo
authors.initials: T
authors.lastname: Klein
mesh_terms: D001921:Brain / Q000033:anatomy & histology* / Q000000981:diagnostic imaging*; D000077321:Deep Learning; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D000069550:Machine Learning; D008279:Magnetic Resonance Imaging / Q000379:methods*; D016571:Neural Networks, Computer; D059906:Neuroimaging / Q000379:methods*
keywords: Brain segmentation; Conditional random field; Convolutional neural networks; Deep learning; Multi-task learning