title: A measure of association for ordered categorical data in population-based studies.
abstract: Ordinal classification scales are commonly used to define a patient's disease status in screening and diagnostic tests such as mammography. Challenges arise in agreement studies when evaluating the association between many raters' classifications of patients' disease or health status when an ordered categorical scale is used. In this paper, we describe a population-based approach and chance-corrected measure of association to evaluate the strength of relationship between multiple raters' ordinal classifications where any number of raters can be accommodated. In contrast to Shrout and Fleiss' intraclass correlation coefficient, the proposed measure of association is invariant with respect to changes in disease prevalence. We demonstrate how unique characteristics of individual raters can be explored using random effects. Simulation studies are conducted to demonstrate the properties of the proposed method under varying assumptions. The methods are applied to two large-scale agreement studies of breast cancer screening and prostate cancer severity.
authors.affiliation: 1 Department of Biostatistics, Boston University, Boston, USA.
authors.forename: Kerrie P
authors.initials: KP
authors.lastname: Nelson
authors.affiliation: 2 Department of Statistics, University of South Carolina, South Carolina, USA.
authors.forename: Don
authors.initials: D
authors.lastname: Edwards
mesh_terms: D056808:Biostatistics / Q000379:methods*; D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D003198:Computer Simulation; D003955:Diagnostic Tests, Routine / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D016014:Linear Models; D008297:Male; D008327:Mammography / Q000706:statistics & numerical data; D008403:Mass Screening / Q000706:statistics & numerical data; D015233:Models, Statistical; D060787:Neoplasm Grading / Q000706:statistics & numerical data; D015588:Observer Variation; D011471:Prostatic Neoplasms / Q000175:diagnosis
keywords: Agreement; association; crossed random effects; generalized linear mixed model; ordinal classifications; weighted kappa