title: The average receiver operating characteristic curve in multireader multicase imaging studies.
abstract: OBJECTIVE
  In multireader, multicase (MRMC) receiver operating characteristic (ROC) studies for evaluating medical imaging systems, the area under the ROC curve (AUC) is often used as a summary metric. Owing to the limitations of AUC, plotting the average ROC curve to accompany the rigorous statistical inference on AUC is recommended. The objective of this article is to investigate methods for generating the average ROC curve from ROC curves of individual readers.
  
  
  METHODS
  We present both a non-parametric method and a parametric method for averaging ROC curves that produce a ROC curve, the area under which is equal to the average AUC of individual readers (a property we call area preserving). We use hypothetical examples, simulated data and a real-world imaging data set to illustrate these methods and their properties.
  
  
  RESULTS
  We show that our proposed methods are area preserving. We also show that the method of averaging the ROC parameters, either the conventional bi-normal parameters (a, b) or the proper bi-normal parameters (c, da), is generally not area preserving and may produce a ROC curve that is intuitively not an average of multiple curves.
  
  
  CONCLUSIONS
  Our proposed methods are useful for making plots of average ROC curves in MRMC studies as a companion to the rigorous statistical inference on the AUC end point. The software implementing these methods is freely available from the authors.
  
  
  CONCLUSIONS
  METHODS for generating the average ROC curve in MRMC ROC studies are formally investigated. The area-preserving criterion we defined is useful to evaluate such methods.
authors.affiliation: Division of Imaging and Applied Mathematics, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA.
authors.forename: W
authors.initials: W
authors.lastname: Chen
authors.forename: F W
authors.initials: FW
authors.lastname: Samuelson
mesh_terms: D019540:Area Under Curve; D001699:Biometry; D003952:Diagnostic Imaging; D006801:Humans; D012372:ROC Curve; D012984:Software