title: A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes.
abstract: OBJECTIVE
  Studies measuring interobserver agreement (reliability) are common in clinical practice, yet discussion of appropriate sample size estimation techniques is minimal as compared with clinical trials. The authors propose a sample size estimation technique to achieve a prespecified lower and upper limit for a confidence interval for the κ coefficient in studies of interobserver agreement.
  
  
  METHODS
  The proposed technique can be used to design a study measuring interobserver agreement with any number of outcomes and any number of raters. Potential application areas include: pathology, psychiatry, dentistry, and physical therapy.
  
  
  RESULTS
  This technique is illustrated using two examples. The first considers a pilot study in oral radiology, whose authors studied the reliability of the mandibular cortical index as measured by three dental professionals. The second example examines the level of interobserver agreement among four nurses with respect to five triage levels used in the Canadian Triage and Acuity Scale.
  
  
  CONCLUSIONS
  This method should be useful in the planning stages of an interobserver agreement study in which the investigator would like to obtain a prespecified level of precision in the estimation of κ. An R software package (R Foundation for Statistical Computing, Vienna, Austria), kappaSize is also provided that implements this method.
authors.affiliation: School of Kinesiology and Health Sciences, York University, Room 364, Norman Bethune College, 4700 Keele Street, Toronto, Ontario, Canada M3J 1P3. mrotondi@yorku.ca
authors.forename: Michael A
authors.initials: MA
authors.lastname: Rotondi
authors.forename: Allan
authors.initials: A
authors.lastname: Donner
mesh_terms: D002170:Canada / Q000453:epidemiology; D016001:Confidence Intervals; D006801:Humans; D008334:Mandible / Q000000981:diagnostic imaging*; D008432:Mathematical Computing; D015588:Observer Variation; D010865:Pilot Projects; D011861:Radiography, Dental / Q000706:statistics & numerical data*; D011862:Radiography, Panoramic / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D018401:Sample Size