Multireader receiver operating characteristic studies: a comparison of study designs.
OBJECTIVE
Traditionally, multireader receiver operating characteristic (ROC) studies have used a "paired-case, paired-reader" design. The statistical power of such a design for inferences about the relative accuracies of the tests was assessed and compared with alternative designs.


METHODS
The noncentrality parameter of an F statistic was used to compute power as a function of the reader and patient sample sizes and the variability and correlation between readings.


RESULTS
For a fixed-power and Type I error rate, the traditional design reduces the number of verified cases required. A hybrid design, in which each reader interprets a different sample of patients, reduces the number of readers, total readings, and reading required per reader. The drawback is a substantial increase in the number of verified cases.


CONCLUSIONS
The ultimate choice of study design depends on the nature of the tests being compared, limiting resources, a priori knowledge of the magnitude of the correlations and variability and logistic complexity.
Obuchowski
N A
NA
Cleveland Clinic Foundation, Department of Biostatistics, OH 44195-5196, USA.
D000704:Analysis of Variance; D003952:Diagnostic Imaging / Q000706:statistics & numerical data*; D006801:Humans; D015233:Models, Statistical; D015588:Observer Variation; D012372:ROC Curve; D011859:Radiography / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D012107:Research Design / Q000706:statistics & numerical data