title: The kappa statistic in reliability studies: use, interpretation, and sample size requirements.
abstract: OBJECTIVE
  This article examines and illustrates the use and interpretation of the kappa statistic in musculoskeletal research.
  
  
  CONCLUSIONS
  The reliability of clinicians' ratings is an important consideration in areas such as diagnosis and the interpretation of examination findings. Often, these ratings lie on a nominal or an ordinal scale. For such data, the kappa coefficient is an appropriate measure of reliability. Kappa is defined, in both weighted and unweighted forms, and its use is illustrated with examples from musculoskeletal research. Factors that can influence the magnitude of kappa (prevalence, bias, and non-independent ratings) are discussed, and ways of evaluating the magnitude of an obtained kappa are considered. The issue of statistical testing of kappa is considered, including the use of confidence intervals, and appropriate sample sizes for reliability studies using kappa are tabulated.
  
  
  CONCLUSIONS
  The article concludes with recommendations for the use and interpretation of kappa.
authors.affiliation: Primary Care Sciences Research Centre, Keele University, Keele, Staffordshire ST5 5BG, United Kingdom. j.sim@keele.ac.uk
authors.forename: Julius
authors.initials: J
authors.lastname: Sim
authors.forename: Chris C
authors.initials: CC
authors.lastname: Wright
mesh_terms: D003627:Data Interpretation, Statistical; D006801:Humans; D015233:Models, Statistical; D009140:Musculoskeletal Diseases / Q000175:diagnosis* / Q000453:epidemiology / Q000503:physiopathology; D015588:Observer Variation; D026761:Physical Therapy Specialty / Q000592:standards*; D015995:Prevalence; D015203:Reproducibility of Results; D012107:Research Design; D018401:Sample Size