title: Interobserver agreement in the evaluation of digitized cervical images.
abstract: OBJECTIVE
  To estimate the agreement among multiple expert colposcopists evaluating high-resolution digitized cervigrams taken from patients with a variety of human papillomavirus (HPV) infection states and previous cervigram interpretations.
  
  
  METHODS
  Twenty expert colposcopists evaluated 939 digitized images of the uterine cervix obtained after the application of 5% acetic acid during the ASCUS-LSIL Triage Study. Twenty images selected to represent a broad range were graded by all the colposcopists. The remaining 919 pictures were distributed by stratified random sampling, such that each image was evaluated by two colposcopists, and each expert evaluated 112 images with similar distributions of cervigram diagnoses and HPV DNA test results. We evaluated interrater agreement among the pairs of colposcopists and confirmed the conclusions using the 20 images they all graded.
  
  
  RESULTS
  Pairs of colposcopists agreed on the diagnosis for only 56.8% of images. Similar agreement was seen regarding number of visible lesions (of low-grade or greater). This variability in ratings remained when the images were stratified by final histologic diagnosis or HPV status. The results were confirmed by the presence of large variability in ratings (ranging in some cases from normal to cancer) for the 20 images graded by all colposcopists.
  
  
  CONCLUSIONS
  Colposcopic diagnosis using static images is poorly reproducible and might reflect similar problems in clinical practice. Researchers should question the use of colposcopic images as a reference standard for teaching and evaluating the presence or severity of disease.
authors: Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, Maryland 20852, USA. guibovij@mail.nih.gov
authors: Jose
authors: J
authors: Jeronimo
authors: L Stewart
authors: LS
authors: Massad
authors: Philip E
authors: PE
authors: Castle
authors: Sholom
authors: S
authors: Wacholder
authors: Mark
authors: M
authors: Schiffman
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D002584:Cervix Uteri / Q000473:pathology* / Q000821:virology; D003127:Colposcopy / Q000592:standards* / Q000706:statistics & numerical data; D005260:Female; D006801:Humans; D046609:Libraries, Digital; D008875:Middle Aged; D015588:Observer Variation; D030361:Papillomavirus Infections / Q000473:pathology*; D002583:Uterine Cervical Neoplasms / Q000473:pathology*