title: The Bethesda Interobserver Reproducibility Study (BIRST): a web-based assessment of the Bethesda 2001 System for classifying cervical cytology.
abstract: BACKGROUND
  The Bethesda System (TBS) along with its companion atlas was updated in 2001 to improve standardization, clarity, and reproducibility of cervical cytology reporting.
  
  
  METHODS
  The authors used a novel web-based format to compare assessments of 77 images demonstrating a range of classical and borderline cytologic changes by a self-selected group of United States cytotechnologists (n = 216) and pathologists (n = 185).
  
  
  RESULTS
  Participants were highly experienced, with 71.2% of cytotechnologists and 53.0% of pathologists reporting >10 years of practice. The mean percentage of exact agreement with the panel was slightly though significantly higher for cytotechnologists (57.0%) compared with pathologists (53.4%), adjusted for experience (P = .004); cervical cytology percentage effort (P = .0005); or cervical accession volume (P = .0002). Compared with the TBS panel, exact agreement was achieved for 55.1% of image ratings compared with 82.3% agreement at the level of Negative vs non-Negative for images with a single-panel interpretation. Agreement with the panel was highest for images classified as Low-Grade Squamous Intraepithelial Lesion and lowest for Atypical Squamous Cells qualified as either of Undetermined Significance or Cannot Exclude a High-Grade Squamous Intraepithelial Lesion. Reviewers were less sensitive in identifying high-grade glandular lesions than they were in identifying high-grade squamous lesions at any threshold (P < .001).
  
  
  CONCLUSIONS
  Morphologic appearances of images were more important determinants than participants' academic or professional degrees with regard to interobserver reproducibility in classifying cervical cytology images. Experienced cytotechnologists and pathologists performed similarly. Participants achieved higher sensitivity for identifying high-grade squamous lesions than they did for high-grade glandular lesions. These findings demonstrated that web-based studies may be useful in assessing interobserver agreement in classifying images.
authors.affiliation: Hormonal and Reproductive Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, Maryland 20852-7374, USA. shermanm@mail.nih.gov
authors.forename: Mark E
authors.initials: ME
authors.lastname: Sherman
authors.forename: Abhijit
authors.initials: A
authors.lastname: Dasgupta
authors.forename: Mark
authors.initials: M
authors.lastname: Schiffman
authors.forename: Ritu
authors.initials: R
authors.lastname: Nayar
authors.forename: Diane
authors.initials: D
authors.lastname: Solomon
mesh_terms: D000230:Adenocarcinoma / Q000145:classification / Q000175:diagnosis / Q000473:pathology; D000716:Anatomy, Artistic; D005260:Female; D006801:Humans; D020407:Internet; D008488:Medical Illustration; D018307:Neoplasms, Squamous Cell / Q000145:classification / Q000175:diagnosis / Q000473:pathology; D015588:Observer Variation; D010336:Pathology / Q000379:methods / Q000592:standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D002583:Uterine Cervical Neoplasms / Q000145:classification* / Q000175:diagnosis / Q000473:pathology*