The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations.
OBJECTIVE
To compare radiologists' performance during interpretation of screening mammograms in the clinic with their performance when reading the same mammograms in a retrospective laboratory study.


METHODS
This study was conducted under an institutional review board-approved, HIPAA-compliant protocol; the need for informed consent was waived. Nine experienced radiologists rated an enriched set of mammograms that they had personally read in the clinic (the "reader-specific" set) mixed with an enriched "common" set of mammograms that none of the participants had previously read in the clinic by using a screening Breast Imaging Reporting and Data System (BI-RADS) rating scale. The original clinical recommendations to recall the women for a diagnostic work-up, for both reader-specific and common sets, were compared with their recommendations during the retrospective experiment. The results are presented in terms of reader-specific and group-averaged sensitivity and specificity levels and the dispersion (spread) of reader-specific performance estimates.


RESULTS
On average, the radiologists' performance was significantly better in the clinic than in the laboratory (P = .035). Interreader dispersion of the computed performance levels was significantly lower during the clinical interpretations (P < .01).


CONCLUSIONS
Retrospective laboratory experiments may not represent either expected performance levels or interreader variability during clinical interpretations of the same set of mammograms in the clinical environment well.
Gur
David
D
Department of Radiology, University of Pittsburgh School of Medicine, 3362 Fifth Ave, Pittsburgh, Pa 15213-31803, USA. gurd@upmc.edu
Bandos
Andriy I
AI
Cohen
Cathy S
CS
Hakim
Christiane M
CM
Hardesty
Lara A
LA
Ganott
Marie A
MA
Perrin
Ronald L
RL
Poller
William R
WR
Shah
Ratan
R
Sumkin
Jules H
JH
Wallace
Luisa P
LP
Rockette
Howard E
HE
D002983:Clinical Competence; D005260:Female; D006801:Humans; D007753:Laboratories; D008327:Mammography / Q000592:standards; D012189:Retrospective Studies; D012680:Sensitivity and Specificity