title: Identifying disagreement in ordinal ratings within a fixed panel of cytotechnologists.
abstract: In many clinical settings, a fixed number of raters screen specimens with more than two ordinal outcomes (for example, mild, moderate, severe). A design is proposed that facilitates the measurement of both interrater and intrarater agreement and associated trends. Design deficiencies are discussed, as are the propriety and interpretation of some common indices of reliability and reproducibility. The concepts are illustrated with data from cytopathologic ratings for sputum light microscopy.
authors.affiliation: Department of Epidemiology/Biomathematics, University of Texas Health Center at Tyler 75710.
authors.forename: D B
authors.initials: DB
authors.lastname: Holiday
authors.forename: J W
authors.initials: JW
authors.lastname: McLarty
mesh_terms: D000818:Animals; D003584:Cytological Techniques; D006801:Humans; D013677:Medical Laboratory Science; D008962:Models, Theoretical; D012680:Sensitivity and Specificity; D013183:Sputum / Q000166:cytology