title: Study design for calibration of clinical examiners measuring periodontal parameters.
abstract: BACKGROUND
  We present an approach to examiner calibration study design where the number of calibration subjects is based on a specified margin of error (half-width of the 95% confidence interval [CI]) of the percentage of agreement (exact and within 1 mm) for both intra- and interexaminer reliability assessments.
  
  
  METHODS
  An experienced standard examiner (S) trained three dental hygienists (A, B, and C) in correct procedures for obtaining a variety of periodontal measures. Duplicate measurements of probing depth (PD [mm]) and the free gingival margin to the cemento-enamel junction (CEJ-GM [mm]) were obtained in a pilot study to design a formal examiner calibration study, where sample sizes were adjusted for the effects of within-subject clustering of binary indices of agreement.
  
  
  RESULTS
  Within-subject clustering of agreement indices resulted in an approximate four-fold increase in the variance of the estimates of percentage of agreement with the standard. PD and CEJ-GM percentage of exact agreement measurements (95% CI) for each examiner-standard pair, respectively, were as follows: AS=55% (48%, 61%) and 70% (62%, 78%); BS=52% (45%, 59%) and 73% (63%, 82%); and CS=55% (50%, 61%) and 72% (65%, 79%). The corresponding 95% CIs unadjusted for the effects of clustering underestimated the margin of error associated with the estimates of exact agreement by as much as 57% for PD and 68% for CEJ-GM.
  
  
  CONCLUSIONS
  Failure to account for dependence among site-level agreement indices results in a false sense of precision in the resulting reliability estimates and can lead to faulty inference.
authors.affiliation: Department of Biostatistics, Bioinformatics, and Epidemiology, Medical University of South Carolina, Charleston, SC 29425, USA. hille@musc.edu
authors.forename: Elizabeth G
authors.initials: EG
authors.lastname: Hill
authors.forename: Elizabeth H
authors.initials: EH
authors.lastname: Slate
authors.forename: Ryan E
authors.initials: RE
authors.lastname: Wiegand
authors.forename: Sara G
authors.initials: SG
authors.lastname: Grossi
authors.forename: Carlos F
authors.initials: CF
authors.lastname: Salinas
mesh_terms: D000293:Adolescent; D000328:Adult; D000704:Analysis of Variance; D003756:Dental Hygienists; D018865:Dental Research / Q000379:methods*; D005260:Female; D006801:Humans; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D015999:Multivariate Analysis; D015588:Observer Variation; D017622:Periodontal Attachment Loss / Q000175:diagnosis*; D010514:Periodontal Pocket / Q000175:diagnosis*; D010865:Pilot Projects; D015203:Reproducibility of Results; D012107:Research Design; D012108:Research Personnel / Q000193:education / Q000592:standards*; D013029:Space-Time Clustering