title: The impact of quality assurance assessment on diffusion tensor imaging outcomes in a large-scale population-based cohort.
abstract: BACKGROUND
  Diffusion tensor imaging (DTI) is applied in investigation of brain biomarkers for neurodevelopmental and neurodegenerative disorders. However, the quality of DTI measurements, like other neuroimaging techniques, is susceptible to several confounding factors (e.g., motion, eddy currents), which have only recently come under scrutiny. These confounds are especially relevant in adolescent samples where data quality may be compromised in ways that confound interpretation of maturation parameters. The current study aims to leverage DTI data from the Philadelphia Neurodevelopmental Cohort (PNC), a sample of 1601 youths with ages of 8-21 who underwent neuroimaging, to: 1) establish quality assurance (QA) metrics for the automatic identification of poor DTI image quality; 2) examine the performance of these QA measures in an external validation sample; 3) document the influence of data quality on developmental patterns of typical DTI metrics.
  
  
  METHODS
  All diffusion-weighted images were acquired on the same scanner. Visual QA was performed on all subjects completing DTI; images were manually categorized as Poor, Good, or Excellent. Four image quality metrics were automatically computed and used to predict manual QA status: Mean voxel intensity outlier count (MEANVOX), Maximum voxel intensity outlier count (MAXVOX), mean relative motion (MOTION) and temporal signal-to-noise ratio (TSNR). Classification accuracy for each metric was calculated as the area under the receiver-operating characteristic curve (AUC). A threshold was generated for each measure that best differentiated visual QA status and applied in a validation sample. The effects of data quality on sensitivity to expected age effects in this developmental sample were then investigated using the traditional MRI diffusion metrics: fractional anisotropy (FA) and mean diffusivity (MD). Finally, our method of QA is compared with DTIPrep.
  
  
  RESULTS
  TSNR (AUC=0.94) best differentiated Poor data from Good and Excellent data. MAXVOX (AUC=0.88) best differentiated Good from Excellent DTI data. At the optimal threshold, 88% of Poor data and 91% Good/Excellent data were correctly identified. Use of these thresholds on a validation dataset (n=374) indicated high accuracy. In the validation sample 83% of Poor data and 94% of Excellent data was identified using thresholds derived from the training sample. Both FA and MD were affected by the inclusion of poor data in an analysis of an age, sex and race matched comparison sample. In addition, we show that the inclusion of poor data results in significant attenuation of the correlation between diffusion metrics (FA and MD) and age during a critical neurodevelopmental period. We find higher correspondence between our QA method and DTIPrep for Poor data, but we find our method to be more robust for apparently high-quality images.
  
  
  CONCLUSIONS
  Automated QA of DTI can facilitate large-scale, high-throughput quality assurance by reliably identifying both scanner and subject induced imaging artifacts. The results present a practical example of the confounding effects of artifacts on DTI analysis in a large population-based sample, and suggest that estimates of data quality should not only be reported but also accounted for in data analysis, especially in studies of development.
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA. Electronic address: roalf@upenn.edu.
authors.forename: David R
authors.initials: DR
authors.lastname: Roalf
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Megan
authors.initials: M
authors.lastname: Quarmley
authors.affiliation: Department of Radiology, University of Pennsylvania, Perelman School of Medicine, USA.
authors.forename: Mark A
authors.initials: MA
authors.lastname: Elliott
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Theodore D
authors.initials: TD
authors.lastname: Satterthwaite
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA; Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA 19104, USA.
authors.forename: Simon N
authors.initials: SN
authors.lastname: Vandekar
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Kosha
authors.initials: K
authors.lastname: Ruparel
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Efstathios D
authors.initials: ED
authors.lastname: Gennatas
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Monica E
authors.initials: ME
authors.lastname: Calkins
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Tyler M
authors.initials: TM
authors.lastname: Moore
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Ryan
authors.initials: R
authors.lastname: Hopson
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Karthik
authors.initials: K
authors.lastname: Prabhakaran
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA.
authors.forename: Chad T
authors.initials: CT
authors.lastname: Jackson
authors.affiliation: Department of Radiology, University of Pennsylvania, Perelman School of Medicine, USA; Section of Biomedical Image Analysis, University of Pennsylvania, USA.
authors.forename: Ragini
authors.initials: R
authors.lastname: Verma
authors.affiliation: Center for Applied Genomics, Children's Hospital of Philadelphia, Philadelphia, PA, USA.
authors.forename: Hakon
authors.initials: H
authors.lastname: Hakonarson
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA; Department of Radiology, University of Pennsylvania, Perelman School of Medicine, USA.
authors.forename: Ruben C
authors.initials: RC
authors.lastname: Gur
authors.affiliation: Neuropsychiatry Section, Department of Psychiatry, USA; Department of Radiology, University of Pennsylvania, Perelman School of Medicine, USA.
authors.forename: Raquel E
authors.initials: RE
authors.lastname: Gur
mesh_terms: D000293:Adolescent; D019540:Area Under Curve; D002648:Child; D015331:Cohort Studies; D056324:Diffusion Tensor Imaging / Q000592:standards*; D005260:Female; D006801:Humans; D007090:Image Interpretation, Computer-Assisted; D008297:Male; D059906:Neuroimaging / Q000592:standards*; D011785:Quality Assurance, Health Care / Q000379:methods*; D012372:ROC Curve; D055815:Young Adult
keywords: Adolescence; Automated quality assurance; Brain maturation; Diffusion tensor imaging; Motion