title: Evaluating accuracy of striatal, pallidal, and thalamic segmentation methods: Comparing automated approaches to manual delineation.
abstract: Accurate automated quantification of subcortical structures is a greatly pursued endeavour in neuroimaging. In an effort to establish the validity and reliability of these methods in defining the striatum, globus pallidus, and thalamus, we investigated differences in volumetry between manual delineation and automated segmentations derived by widely used FreeSurfer and FSL packages, and a more recent segmentation method, the MAGeT-Brain algorithm. In a first set of experiments, the basal ganglia and thalamus of thirty subjects (15 first episode psychosis [FEP], 15 controls) were manually defined and compared to the labels generated by the three automated methods. Our results suggest that all methods overestimate volumes compared to the manually derived "gold standard", with the least pronounced differences produced using MAGeT. The least between-method variability was noted for the striatum, whereas marked differences between manual segmentation and MAGeT compared to FreeSurfer and FSL emerged for the globus pallidus and thalamus. Correlations between manual segmentation and automated methods were strongest for MAGeT (range: 0.51 to 0.92; p<0.01, corrected), whereas FreeSurfer and FSL showed moderate to strong Pearson correlations (range 0.44-0.86; p<0.05, corrected), with the exception of FreeSurfer pallidal (r=0.31, p=0.10) and FSL thalamic segmentations (r=0.37, p=0.051). Bland-Altman plots highlighted a tendency for greater volumetric differences between manual labels and automated methods at the lower end of the distribution (i.e. smaller structures), which was most prominent for bilateral thalamus across automated pipelines, and left globus pallidus for FSL. We then went on to examine volume and shape of the basal ganglia structures using automated techniques in 135 FEP patients and 88 controls. The striatum and globus pallidus were significantly larger in FEP patients compared to controls bilaterally, irrespective of the method used. MAGeT-Brain was more sensitive to shape-based group differences, and uncovered widespread surface expansions in the striatum and globus pallidus bilaterally in FEP patients compared to controls, and surface contractions in bilateral thalamus (FDR-corrected). By contrast, after using a recommended cluster-wise thresholding method, FSL only detected differences in the right ventral striatum (FEP>Control) and one cluster of the left thalamus (Control>FEP). These results suggest that different automated pipelines segment subcortical structures with varying degrees of variability compared to manual methods, with particularly pronounced differences found with FreeSurfer and FSL for the globus pallidus and thalamus.
authors.affiliation: McConnell Brain Imaging Centre, Montreal Neurological Institute, Montreal, Canada; Douglas Mental Health University Institute, McGill University, Montreal, Canada; McGill Centre for Integrative Neuroscience, McGill University, Montreal, Canada; Ludmer Centre for Neuroinformatics and Mental Health, Montreal, Canada. Electronic address: carolina.makowski@mail.mcgill.ca.
authors.forename: Carolina
authors.initials: C
authors.lastname: Makowski
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada.
authors.forename: Sophie
authors.initials: S
authors.lastname: Béland
authors.affiliation: McConnell Brain Imaging Centre, Montreal Neurological Institute, Montreal, Canada; McGill Centre for Integrative Neuroscience, McGill University, Montreal, Canada; Ludmer Centre for Neuroinformatics and Mental Health, Montreal, Canada.
authors.forename: Penelope
authors.initials: P
authors.lastname: Kostopoulos
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Institute of Biomaterials and Biomedical Engineering, University of Toronto, Toronto, Canada.
authors.forename: Nikhil
authors.initials: N
authors.lastname: Bhagwat
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Department of Psychiatry, McGill University, Montreal, Canada.
authors.forename: Gabriel A
authors.initials: GA
authors.lastname: Devenyi
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Department of Psychiatry, McGill University, Montreal, Canada.
authors.forename: Ashok K
authors.initials: AK
authors.lastname: Malla
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Department of Psychiatry, McGill University, Montreal, Canada.
authors.forename: Ridha
authors.initials: R
authors.lastname: Joober
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Department of Psychiatry, McGill University, Montreal, Canada.
authors.forename: Martin
authors.initials: M
authors.lastname: Lepage
authors.affiliation: Douglas Mental Health University Institute, McGill University, Montreal, Canada; Department of Psychiatry, McGill University, Montreal, Canada; Department of Biological and Biomedical Engineering, McGill University, Montreal, Canada. Electronic address: mallar@cobralab.ca.
authors.forename: M Mallar
authors.initials: MM
authors.lastname: Chakravarty
mesh_terms: D000328:Adult; D003342:Corpus Striatum / Q000033:anatomy & histology / Q000000981:diagnostic imaging*; D005260:Female; D005917:Globus Pallidus / Q000033:anatomy & histology / Q000000981:diagnostic imaging*; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods / Q000592:standards*; D008279:Magnetic Resonance Imaging / Q000379:methods / Q000592:standards*; D008297:Male; D059906:Neuroimaging / Q000379:methods / Q000592:standards*; D011618:Psychotic Disorders / Q000000981:diagnostic imaging* / Q000473:pathology; D015203:Reproducibility of Results; D013788:Thalamus / Q000033:anatomy & histology / Q000000981:diagnostic imaging*; D055815:Young Adult
keywords: Automated segmentation; First episode psychosis; Manual delineation; Shape morphometry; Structural MRI; Subcortical structure