title: Estimating and comparing diagnostic tests' accuracy when the gold standard is not binary.
abstract: OBJECTIVE
  Investigators often need to assess the accuracies of diagnostic tests when the gold standard is not binary-scale. The objective of this article is to describe nonparametric estimators of diagnostic test accuracy when the gold standard is continuous, ordinal, and nominal scale.
  
  
  METHODS
  A nonparametric method of estimating and comparing the area under receiver operating characteristic (ROC) curves, proposed by DeLong et al, is extended to situations in which the gold standard is not binary. Two examples illustrate the methods.
  
  
  RESULTS
  Measures of diagnostic test accuracy, their variance, and tests for comparing two diagnostic tests' accuracies in paired designs are presented for situations in which the gold standard is continuous, ordinal, and nominal scale. These summary measures of diagnostic test accuracy are analogous in form and interpretation to the area under the ROC curve.
  
  
  CONCLUSIONS
  Dichotomizing the outcomes of a gold standard so that traditional ROC methods can be applied can lead to bias. The methods described here are useful for assessing and comparing summary test accuracy when the gold standard is not binary scale. They have limitations similar to other summary indices.
authors.affiliation: Department of Quantitative Health Sciences/Wb4, The Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195, USA. nobuchow@bio.ri.ccf.org
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Obuchowski
mesh_terms: D003952:Diagnostic Imaging / Q000592:standards*; D006801:Humans; D007680:Kidney Neoplasms / Q000000981:diagnostic imaging; D008279:Magnetic Resonance Imaging; D015233:Models, Statistical; D009203:Myocardial Infarction / Q000175:diagnosis; D011237:Predictive Value of Tests; D012372:ROC Curve; D018709:Statistics, Nonparametric; D014057:Tomography, X-Ray Computed