title: Teaching receiver operating characteristic analysis: an interactive laboratory exercise.
abstract: OBJECTIVE
  Despite its fundamental importance in the evaluation of diagnostic tests, receiver operating characteristic (ROC) analysis is not easily understood. The purpose of this project was to create a learning experience that resulted in an intuitive understanding of the basic principles of ROC analysis.
  
  
  METHODS
  An interactive laboratory exercise was developed for a class about radiology testing taught within a clinical epidemiology course between 2000 and 2009. The physician students in the course were clinical fellows from various medical specialties who were enrolled in a graduate degree program in clinical investigation. For the exercise, the class was divided into six groups. Each group interpreted radiographs from a set of 50 exams of the peripheral skeleton to determine the presence or absence of an acute fracture. Data from the class were pooled and given to each student. Students calculated the area under the ROC curve (AUC) corresponding to overall class performance. A binormal ROC curve was also fitted to the data from each class year.
  
  
  RESULTS
  The laboratory exercise was conducted for 8 years with approximately 20-30 students per year. The mean AUC over the eight laboratory classes was 0.72 with a standard deviation of 0.08 (range, 0.60-0.85).
  
  
  CONCLUSIONS
  With some simplifications in design, an observer study can be conducted in a laboratory classroom setting. Participatory data collection promotes the intuitive understanding of ROC analysis principles.
authors.affiliation: Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, 600 North Wolfe Street, Baltimore, MD 21287, USA. jeng@jhmi.edu
authors.forename: John
authors.initials: J
authors.lastname: Eng
mesh_terms: D019540:Area Under Curve; D006801:Humans; D007396:Internship and Residency; D012372:ROC Curve; D011857:Radiographic Image Interpretation, Computer-Assisted; D011871:Radiology / Q000193:education*; D012680:Sensitivity and Specificity; D013663:Teaching / Q000379:methods*