title: Modeling agreement on categorical scales in the presence of random scorers.
abstract: Kappa coefficients are often used to assess agreement between two fixed scorers on categorical scales. Cohen's version is popular for nominal scales and the weighted version for ordinal scales. In the present paper, similar agreement coefficients are defined for random scorers. A partial-Bayesian methodology is then developed to directly relate these agreement coefficients to predictors through a multilevel model. Statistical properties of the proposed approach are studied using simulations. Finally, the approach is applied to gynecological and medical imaging data.
authors.affiliation: Department of Methodology and Statistics, CAPHRI, Maastricht University, P. Debyeplein 1, 6229 HA Maastricht, The Netherlands sophie.vanbelle@maastrichtuniversity.nl.
authors.forename: Sophie
authors.initials: S
authors.lastname: Vanbelle
authors.affiliation: Leuven Biostatistics and Statistical Bioinformatics Centre, KU Leuven, Kapucijnenvoer 35, 3000 Leuven, Belgium.
authors.forename: Emmanuel
authors.initials: E
authors.lastname: Lesaffre
mesh_terms: D001499:Bayes Theorem; D005260:Female; D006801:Humans; D008390:Markov Chains; D015233:Models, Statistical; D011655:Pulmonary Embolism / Q000175:diagnosis*; D015203:Reproducibility of Results; D002583:Uterine Cervical Neoplasms / Q000175:diagnosis*
keywords: Markov chain Monte Carlo; Multilevel; Nested; Rater; Reliability