title: A global goodness-of-fit test for receiver operating characteristic curve analysis via the bootstrap method.
abstract: OBJECTIVE
  Medical classification accuracy studies often yield continuous data based on predictive models for treatment outcomes. A popular method for evaluating the performance of diagnostic tests is the receiver operating characteristic (ROC) curve analysis. The main objective was to develop a global statistical hypothesis test for assessing the goodness-of-fit (GOF) for parametric ROC curves via the bootstrap.
  
  
  METHODS
  A simple log (or logit) and a more flexible Box-Cox normality transformations were applied to untransformed or transformed data from two clinical studies to predict complications following percutaneous coronary interventions (PCIs) and for image-guided neurosurgical resection results predicted by tumor volume, respectively. We compared a non-parametric with a parametric binormal estimate of the underlying ROC curve. To construct such a GOF test, we used the non-parametric and parametric areas under the curve (AUCs) as the metrics, with a resulting p value reported.
  
  
  RESULTS
  In the interventional cardiology example, logit and Box-Cox transformations of the predictive probabilities led to satisfactory AUCs (AUC=0.888; p=0.78, and AUC=0.888; p=0.73, respectively), while in the brain tumor resection example, log and Box-Cox transformations of the tumor size also led to satisfactory AUCs (AUC=0.898; p=0.61, and AUC=0.899; p=0.42, respectively). In contrast, significant departures from GOF were observed without applying any transformation prior to assuming a binormal model (AUC=0.766; p=0.004, and AUC=0.831; p=0.03), respectively.
  
  
  CONCLUSIONS
  In both studies the p values suggested that transformations were important to consider before applying any binormal model to estimate the AUC. Our analyses also demonstrated and confirmed the predictive values of different classifiers for determining the interventional complications following PCIs and resection outcomes in image-guided neurosurgery.
authors.affiliation: Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, MIT, MA, USA. zou@bwh.harvard.edu
authors.forename: Kelly H
authors.initials: KH
authors.lastname: Zou
authors.forename: Frederic S
authors.initials: FS
authors.lastname: Resnic
authors.forename: Ion-Florin
authors.initials: IF
authors.lastname: Talos
authors.forename: Daniel
authors.initials: D
authors.lastname: Goldberg-Zimring
authors.forename: Jui G
authors.initials: JG
authors.lastname: Bhagwat
authors.forename: Steven J
authors.initials: SJ
authors.lastname: Haker
authors.forename: Ron
authors.initials: R
authors.lastname: Kikinis
authors.forename: Ferenc A
authors.initials: FA
authors.lastname: Jolesz
authors.forename: Lucila
authors.initials: L
authors.lastname: Ohno-Machado
mesh_terms: D000293:Adolescent; D000328:Adult; D000465:Algorithms; D015906:Angioplasty, Balloon, Coronary / Q000401:mortality*; D001932:Brain Neoplasms / Q000401:mortality* / Q000601:surgery; D002138:Calibration; D003627:Data Interpretation, Statistical; D020000:Decision Support Systems, Clinical; D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D016002:Discriminant Analysis; D005103:Expert Systems; D005260:Female; D006801:Humans; D015994:Incidence; D008297:Male; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000379:methods*; D011379:Prognosis; D012372:ROC Curve; D018570:Risk Assessment / Q000379:methods*; D012307:Risk Factors; D016019:Survival Analysis; D015996:Survival Rate; D014481:United States / Q000453:epidemiology