title: The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations.
abstract: OBJECTIVE
  To compare radiologists' performance during interpretation of screening mammograms in the clinic with their performance when reading the same mammograms in a retrospective laboratory study.
  
  
  METHODS
  This study was conducted under an institutional review board-approved, HIPAA-compliant protocol; the need for informed consent was waived. Nine experienced radiologists rated an enriched set of mammograms that they had personally read in the clinic (the "reader-specific" set) mixed with an enriched "common" set of mammograms that none of the participants had previously read in the clinic by using a screening Breast Imaging Reporting and Data System (BI-RADS) rating scale. The original clinical recommendations to recall the women for a diagnostic work-up, for both reader-specific and common sets, were compared with their recommendations during the retrospective experiment. The results are presented in terms of reader-specific and group-averaged sensitivity and specificity levels and the dispersion (spread) of reader-specific performance estimates.
  
  
  RESULTS
  On average, the radiologists' performance was significantly better in the clinic than in the laboratory (P = .035). Interreader dispersion of the computed performance levels was significantly lower during the clinical interpretations (P < .01).
  
  
  CONCLUSIONS
  Retrospective laboratory experiments may not represent either expected performance levels or interreader variability during clinical interpretations of the same set of mammograms in the clinical environment well.
authors.affiliation: Department of Radiology, University of Pittsburgh School of Medicine, 3362 Fifth Ave, Pittsburgh, Pa 15213-31803, USA. gurd@upmc.edu
authors.forename: David
authors.initials: D
authors.lastname: Gur
authors.forename: Andriy I
authors.initials: AI
authors.lastname: Bandos
authors.forename: Cathy S
authors.initials: CS
authors.lastname: Cohen
authors.forename: Christiane M
authors.initials: CM
authors.lastname: Hakim
authors.forename: Lara A
authors.initials: LA
authors.lastname: Hardesty
authors.forename: Marie A
authors.initials: MA
authors.lastname: Ganott
authors.forename: Ronald L
authors.initials: RL
authors.lastname: Perrin
authors.forename: William R
authors.initials: WR
authors.lastname: Poller
authors.forename: Ratan
authors.initials: R
authors.lastname: Shah
authors.forename: Jules H
authors.initials: JH
authors.lastname: Sumkin
authors.forename: Luisa P
authors.initials: LP
authors.lastname: Wallace
authors.forename: Howard E
authors.initials: HE
authors.lastname: Rockette
mesh_terms: D002983:Clinical Competence; D005260:Female; D006801:Humans; D007753:Laboratories; D008327:Mammography / Q000592:standards; D012189:Retrospective Studies; D012680:Sensitivity and Specificity