title: Clinicians are right not to like Cohen's κ.
abstract: Clinicians are interested in observer variation in terms of the probability of other raters (interobserver) or themselves (intraobserver) obtaining the same answer. Cohen's κ is commonly used in the medical literature to express such agreement in categorical outcomes. The value of Cohen's κ, however, is not sufficiently informative because it is a relative measure, while the clinician's question of observer variation calls for an absolute measure. Using an example in which the observed agreement and κ lead to different conclusions, we illustrate that percentage agreement is an absolute measure (a measure of agreement) and that κ is a relative measure (a measure of reliability). For the data to be useful for clinicians, measures of agreement should be used. The proportion of specific agreement, expressing the agreement separately for the positive and the negative ratings, is the most appropriate measure for conveying the relevant information in a 2 × 2 table and is most informative for clinicians.
authors.affiliation: Department of Epidemiology and Biostatistics, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, Netherlands. hcw.devet@vumc.nl
authors.forename: Henrica C W
authors.initials: HC
authors.lastname: de Vet
authors.forename: Lidwine B
authors.initials: LB
authors.lastname: Mokkink
authors.forename: Caroline B
authors.initials: CB
authors.lastname: Terwee
authors.forename: Otto S
authors.initials: OS
authors.lastname: Hoekstra
authors.forename: Dirk L
authors.initials: DL
authors.lastname: Knol
mesh_terms: D006801:Humans; D015588:Observer Variation; D011336:Probability; D015203:Reproducibility of Results; D012107:Research Design