title: Multireader receiver operating characteristic studies: a comparison of study designs.
abstract: OBJECTIVE
  Traditionally, multireader receiver operating characteristic (ROC) studies have used a "paired-case, paired-reader" design. The statistical power of such a design for inferences about the relative accuracies of the tests was assessed and compared with alternative designs.
  
  
  METHODS
  The noncentrality parameter of an F statistic was used to compute power as a function of the reader and patient sample sizes and the variability and correlation between readings.
  
  
  RESULTS
  For a fixed-power and Type I error rate, the traditional design reduces the number of verified cases required. A hybrid design, in which each reader interprets a different sample of patients, reduces the number of readers, total readings, and reading required per reader. The drawback is a substantial increase in the number of verified cases.
  
  
  CONCLUSIONS
  The ultimate choice of study design depends on the nature of the tests being compared, limiting resources, a priori knowledge of the magnitude of the correlations and variability and logistic complexity.
authors.affiliation: Cleveland Clinic Foundation, Department of Biostatistics, OH 44195-5196, USA.
authors.forename: N A
authors.initials: NA
authors.lastname: Obuchowski
mesh_terms: D000704:Analysis of Variance; D003952:Diagnostic Imaging / Q000706:statistics & numerical data*; D006801:Humans; D015233:Models, Statistical; D015588:Observer Variation; D012372:ROC Curve; D011859:Radiography / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D012107:Research Design / Q000706:statistics & numerical data