Weighted least-squares approach for comparing correlated kappa.
In the medical sciences, studies are often designed to assess the agreement between different raters or different instruments. The kappa coefficient is a popular index of agreement for binary and categorical ratings. Here we focus on testing for the equality of two dependent kappa coefficients. We use the weighted least-squares (WLS) approach of Koch et al. (1977, Biometrics 33, 133-158) to take into account the correlation between the estimated kappa statistics. We demonstrate how the SAS PROC CATMOD can be used to test for the equality of dependent Cohen's kappa coefficients and dependent intraclass kappa coefficients with nominal categorical ratings. We also test for the equality of dependent Cohen's kappa and dependent weighted kappa with ordinal ratings. The major advantage of the WLS approach is that it allows the data analyst a way of testing dependent kappa with popular SAS software. The WLS approach can handle any number of categories. Analyses of three biomedical studies are used for illustration.
Barnhart
Huiman X
HX
Department of Biostatistics, The Rollins School of Public Health of Emory University, 1518 Clifton Road, NE, Atlanta, Georgia 30322, USA. hbarnha@sph.emory.edu
Williamson
John M
JM
D001284:Atrophy / Q000175:diagnosis; D015982:Bias; D001699:Biometry / Q000379:methods*; D005128:Eye Diseases / Q000175:diagnosis; D005260:Female; D006801:Humans; D016018:Least-Squares Analysis; D015588:Observer Variation; D015203:Reproducibility of Results; D002577:Uterine Cervical Diseases / Q000175:diagnosis