title: The evaluation of preprocessing choices in single-subject BOLD fMRI using NPAIRS performance metrics.
abstract: This work proposes an alternative to simulation-based receiver operating characteristic (ROC) analysis for assessment of fMRI data analysis methodologies. Specifically, we apply the rapidly developing nonparametric prediction, activation, influence, and reproducibility resampling (NPAIRS) framework to obtain cross-validation-based model performance estimates of prediction accuracy and global reproducibility for various degrees of model complexity. We rely on the concept of an analysis chain meta-model in which all parameters of the preprocessing steps along with the final statistical model are treated as estimated model parameters. Our ROC analog, then, consists of plotting prediction vs. reproducibility results as curves of model complexity for competing meta-models. Two theoretical underpinnings are crucial to utilizing this new validation technique. First, we explore the relationship between global signal-to-noise and our reproducibility estimates as derived previously. Second, we submit our model complexity curves in the prediction versus reproducibility space as reflecting classic bias-variance tradeoffs. Among the particular analysis chains considered, we found little impact in performance metrics with alignment, some benefit with temporal detrending, and greatest improvement with spatial smoothing.
authors.affiliation: Biomedical Engineering, University of Minnesota, Minneapolis, Minnesota 55455, USA.
authors.forename: Stephen
authors.initials: S
authors.lastname: LaConte
authors.forename: Jon
authors.initials: J
authors.lastname: Anderson
authors.forename: Suraj
authors.initials: S
authors.lastname: Muley
authors.forename: James
authors.initials: J
authors.lastname: Ashe
authors.forename: Sally
authors.initials: S
authors.lastname: Frutiger
authors.forename: Kelly
authors.initials: K
authors.lastname: Rehm
authors.forename: Lars Kai
authors.initials: LK
authors.lastname: Hansen
authors.forename: Essa
authors.initials: E
authors.lastname: Yacoub
authors.forename: Xiaoping
authors.initials: X
authors.lastname: Hu
authors.forename: David
authors.initials: D
authors.lastname: Rottenberg
authors.forename: Stephen
authors.initials: S
authors.lastname: Strother
mesh_terms: D000328:Adult; D000465:Algorithms; D001931:Brain Mapping / Q000379:methods; D002540:Cerebral Cortex / Q000502:physiology*; D001330:Electronic Data Processing / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000706:statistics & numerical data; D021621:Imaging, Three-Dimensional / Q000706:statistics & numerical data; D008279:Magnetic Resonance Imaging / Q000706:statistics & numerical data*; D008297:Male; D010100:Oxygen / Q000097:blood*; D010101:Oxygen Consumption / Q000502:physiology; D011597:Psychomotor Performance / Q000502:physiology*; D012372:ROC Curve; D015203:Reproducibility of Results; D012984:Software / Q000706:statistics & numerical data
chemical_list: D010100:Oxygen