The Bethesda Interobserver Reproducibility Study (BIRST): a web-based assessment of the Bethesda 2001 System for classifying cervical cytology.
BACKGROUND
The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting.


METHODS
The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185).


RESULTS
Participants were highly experienced, with 71.2% of cytotechnologists and 53.0% of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0%) compared with pathologists (53.4%), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1% of image ratings compared with 82.3% agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001).


CONCLUSIONS
Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.
Sherman
Mark E
ME
Hormonal and Reproductive Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, Maryland 20852-7374, USA. shermanm@mail.nih.gov
Dasgupta
Abhijit
A
Schiffman
Mark
M
Nayar
Ritu
R
Solomon
Diane
D
D000230:Adenocarcinoma / Q000145:classification / Q000175:diagnosis / Q000473:pathology; D000716:Anatomy, Artistic; D005260:Female; D006801:Humans; D020407:Internet; D008488:Medical Illustration; D018307:Neoplasms, Squamous Cell / Q000145:classification / Q000175:diagnosis / Q000473:pathology; D015588:Observer Variation; D010336:Pathology / Q000379:methods / Q000592:standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D002583:Uterine Cervical Neoplasms / Q000145:classification* / Q000175:diagnosis / Q000473:pathology*