title: Agreement between cytotechnologists and cytopathologists as a new measure of cytopathologist performance in gynecologic cytology.
abstract: BACKGROUND
  Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions.
  
  
  METHODS
  Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10% of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs.
  
  
  RESULTS
  This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4%) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2%]).
  
  
  CONCLUSIONS
  Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance. Cancer Cytopathol 2017;125:576-80. © 2017 American Cancer Society.
authors.affiliation: Department of Pathology, University of Texas Southwestern Medical Center, Dallas, Texas.
authors.forename: Andrew M
authors.identifier: http://orcid.org/0000-0003-2076-2304
authors.initials: AM
authors.lastname: Quinn
authors.affiliation: Division of Biostatistics, Department of Clinical Sciences, University of Texas Southwestern Medical Center, Dallas, Texas.
authors.forename: Abu T
authors.initials: AT
authors.lastname: Minhajuddin
authors.affiliation: Division of Biostatistics, Department of Clinical Sciences, University of Texas Southwestern Medical Center, Dallas, Texas.
authors.forename: Linda S
authors.initials: LS
authors.lastname: Hynan
authors.affiliation: Division of Biostatistics, Department of Clinical Sciences, University of Texas Southwestern Medical Center, Dallas, Texas.
authors.forename: Joan S
authors.initials: JS
authors.lastname: Reisch
authors.affiliation: Department of Pathology, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.
authors.forename: Edmund S
authors.identifier: http://orcid.org/0000-0003-1987-2004
authors.initials: ES
authors.lastname: Cibas
mesh_terms: D001707:Biopsy, Needle; D002294:Carcinoma, Squamous Cell / Q000473:pathology*; D003581:Cytodiagnosis / Q000379:methods; D005260:Female; D005833:Genital Neoplasms, Female / Q000473:pathology*; D006801:Humans; D008297:Male; D017056:Medical Laboratory Personnel / Q000592:standards*; D060787:Neoplasm Grading / Q000379:methods; D009367:Neoplasm Staging / Q000379:methods; D010338:Pathology, Clinical / Q000592:standards*; D011361:Professional Competence; D011785:Quality Assurance, Health Care; D014626:Vaginal Smears
keywords: Papanicolaou test; SQL; agreement; cytopathologist; cytotechnologist; gynecologic; outlier; performance; quality; κ