The dependence of Cohen's kappa on the prevalence does not matter.
OBJECTIVE
The dependence of Cohen's kappa on the prevalence has been a major concern in the literature. Indeed, it indicates a serious limitation with respect to comparing kappa-values among studies with varying prevalences.


METHODS
The basic arguments used by different authors are reviewed.


RESULTS
Two types of dependence can be distinguished: a dependence on the observed marginal prevalences and a dependence on the prevalence of a latent binary variable, representing the true status. The first dependence is simply a consequence of the purpose of kappa, which is to improve the interpretation of agreement rates, and so does not constitute a real argument against kappa. The second occurs only if one can change the prevalence without changing sensitivity and specificity. Typically, in agreement studies a change in prevalence implies also a change in sensitivity and specificity, and we show that in such a framework the dependence on the prevalence becomes negligible.


CONCLUSIONS
We should stop criticizing kappa for its dependence on the prevalence. Instead, we should focus on its dependence on the composition of the population with respect to subjects easy or difficult to agree on.
Vach
Werner
W
Department of Statistics, University of Southern Denmark, Campusvej 55, 5230 Odense M, Denmark. werner.v@stat.sdu.dk
D003627:Data Interpretation, Statistical; D006801:Humans; D015233:Models, Statistical; D018579:Patient Selection; D015995:Prevalence; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity