title: Statistical methods for MRMC ROC studies.
abstract: OBJECTIVE
  In radiology, multireader, multicase (MRMC) receiver-operating characteristic studies are commonly used to evaluate the accuracy of diagnostic imaging systems. The special feature of an MRMC receiver-operating characteristic study requires that the same set of patients' images be examined by the same set of doctors. One main difficulty of analyzing MRMC data is a complicated correlation structure. Four commonly used methods are available for dealing with this complicated correlation structure. The authors conducted an extensive simulation study to assess the performance of these methods in finite sample sizes. They summarize the relative strengths and weaknesses of these methods and make recommendations on the use of these methods.
  
  
  METHODS
  A comprehensive simulation study was conducted to assess finite-sample performance of these methods with continuous data. The use of these methods for magnetic resonance imaging to predict extracapsular extension of prostate gland tumors is also illustrated.
  
  
  RESULTS
  The results indicate that when test outcomes are continuous, all four methods perform well for estimating the difference in areas under the curves for two diagnostic tests. On the basis of these results, it seems that any of these approaches is appropriate for analyzing an MRMC data set with continuous or pseudocontinuous data.
  
  
  CONCLUSIONS
  The Dorfman-Berbaum-Metz method is the most practical analysis method to implement in a wide variety of scenarios. Also, in MRMC studies, radiologists should be encouraged to use the entire rating scale rather than tending toward a binary "diseased" or "not diseased" decision.
authors.affiliation: HSR&D Center of Excellence, VA Puget Sound Health Care System, Seattle, WA 98101, USA.
authors.forename: Annie
authors.initials: A
authors.lastname: Skaron
authors.forename: Kang
authors.initials: K
authors.lastname: Li
authors.forename: Xiao-Hua
authors.initials: XH
authors.lastname: Zhou
mesh_terms: D000704:Analysis of Variance; D016001:Confidence Intervals; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D015588:Observer Variation; D019047:Phantoms, Imaging; D011471:Prostatic Neoplasms / Q000473:pathology*; D012372:ROC Curve; D011871:Radiology; D012107:Research Design