title: Independent versus sequential reading in ROC studies of computer-assist modalities: analysis of components of variance.
abstract: OBJECTIVE
  The authors analyzed two methods for arranging the temporal sequencing of unaided versus computer-assisted reading in multiple-reader, multiple-case receiver operating characteristic studies of detection of solitary pulmonary nodules on chest radiographs. In the "independent" mode the readings are separated by about I month; in the "sequential" mode, the computer-assisted reading immediately follows the unassisted reading.
  
  
  METHODS
  The authors used the method of Beiden, Wagner, and Campbell (BWC) to decompose the components of variance of receiver operating characteristic accuracy measures into those that are correlated and those that are uncorrelated across reading conditions. Only the latter contribute to uncertainty in estimates of the difference in accuracy measures across reading conditions (unaided vs aided). This method was used to analyze data from two independent studies of the detection of solitary pulmonary nodules on chest radiographs.
  
  
  RESULTS
  In the sequential reading mode the components that were correlated across reading conditions increased compared to the independent reading mode, as might be expected. What was not anticipated was the fact that the total reader variance was approximately the same for the two reading modes. The results were remarkably similar across the two independent studies analyzed.
  
  
  CONCLUSIONS
  The sequential reading mode may thus be the more sensitive probe of the difference between unassisted and computer-assisted reading, if the mean effect is unperturbed (as here). It is also the least demanding on the logistics and investment of reader time.
authors.affiliation: Center for Devices and Radiological Health, Food and Drug Administration, Rockville, MD 20857, USA.
authors.forename: Sergey V
authors.initials: SV
authors.lastname: Beiden
authors.forename: Robert F
authors.initials: RF
authors.lastname: Wagner
authors.forename: Kunio
authors.initials: K
authors.lastname: Doi
authors.forename: Robert M
authors.initials: RM
authors.lastname: Nishikawa
authors.forename: Matthew
authors.initials: M
authors.lastname: Freedman
authors.forename: Shih-Chung Ben
authors.initials: SC
authors.lastname: Lo
authors.forename: Xin-Wei
authors.initials: XW
authors.lastname: Xu
mesh_terms: D000704:Analysis of Variance; D003936:Diagnosis, Computer-Assisted; D006801:Humans; D012372:ROC Curve; D013902:Radiography, Thoracic; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging*