title: Diffusion Capillary Phantom vs. Human Data: Outcomes for Reconstruction Methods Depend on Evaluation Medium.
abstract: OBJECTIVE
  Diffusion MRI provides a non-invasive way of estimating structural connectivity in the brain. Many studies have used diffusion phantoms as benchmarks to assess the performance of different tractography reconstruction algorithms and assumed that the results can be applied to in vivo studies. Here we examined whether quality metrics derived from a common, publically available, diffusion phantom can reliably predict tractography performance in human white matter tissue.
  
  
  METHODS
  We compared estimates of fiber length and fiber crossing among a simple tensor model (diffusion tensor imaging), a more complicated model (ball-and-sticks) and model-free (diffusion spectrum imaging, generalized q-sampling imaging) reconstruction methods using a capillary phantom and in vivo human data (N = 14).
  
  
  RESULTS
  Our analysis showed that evaluation outcomes differ depending on whether they were obtained from phantom or human data. Specifically, the diffusion phantom favored a more complicated model over a simple tensor model or model-free methods for resolving crossing fibers. On the other hand, the human studies showed the opposite pattern of results, with the model-free methods being more advantageous than model-based methods or simple tensor models. This performance difference was consistent across several metrics, including estimating fiber length and resolving fiber crossings in established white matter pathways.
  
  
  CONCLUSIONS
  These findings indicate that the construction of current capillary diffusion phantoms tends to favor complicated reconstruction models over a simple tensor model or model-free methods, whereas the in vivo data tends to produce opposite results. This brings into question the previous phantom-based evaluation approaches and suggests that a more realistic phantom or simulation is necessary to accurately predict the relative performance of different tractography reconstruction methods.
authors.affiliation: Department of Psychology, Center for the Neural Basis of Cognition, University of Pittsburgh Pittsburgh, PA, USA.
authors.forename: Sarah D
authors.initials: SD
authors.lastname: Lichenstein
authors.affiliation: Department of Neurological Sciences, University of Vermont Burlington, VT, USA.
authors.forename: James H
authors.initials: JH
authors.lastname: Bishop
authors.affiliation: Department of Psychology, Center for the Neural Basis of Cognition, Carnegie Mellon University Pittsburgh, PA, USA.
authors.forename: Timothy D
authors.initials: TD
authors.lastname: Verstynen
authors.affiliation: Department of Neurological Surgery, University of Pittsburgh Pittsburgh, PA, USA.
authors.forename: Fang-Cheng
authors.initials: FC
authors.lastname: Yeh
keywords: diffusion MRI; human brain; phantom; reconstruction; tractography