Evaluating agreement: conducting a reliability study.
Instruments that are useful in clinical or research practice will, when the object of measurement is stable, yield similar results when applied at different times, in different situations, or by different users. Studies that measure the relation of differences between patients or subjects and measurement error (reliability studies) are becoming increasingly common in the orthopaedic literature. In this paper, we identify common aspects of reliability studies and suggest features that improve the reader's confidence in the results. One concept serves as the foundation for all further consideration: in order for a reliability study to be relevant, the patients, raters, and test administration in the study must be similar to the clinical or research context in which the instrument will be used. We introduce the statistical measures that readers will most commonly encounter in reliability studies, and we suggest an approach to sample-size estimation. Readers interested in critically appraising reliability studies or in developing their own reliability studies may find this review helpful.
Department of Clinical Epidemiology and Biostatistics, McMaster University, 293 Wellington Street North, Suite 110, Hamilton, ON L8L 2X2, Canada.
Paul J
PJ
Karanicolas
Mohit
M
Bhandari
Hans
H
Kreder
Antonio
A
Moroni
Martin
M
Richardson
Stephen D
SD
Walter
Geoff R
GR
Norman
Gordon H
GH
Guyatt
D035843:Biomedical Research / Q000379:methods*; D006801:Humans; D009985:Orthopedics; D015203:Reproducibility of Results; D012107:Research Design; D018401:Sample Size; D013223:Statistics as Topic