A comparison of face-to-face and remote assessment of inter-rater reliability on the Hamilton Depression Rating Scale via videoconferencing.
Poor inter-rater reliability (IRR) is an important methodological factor that may contribute to failed trials. The sheer number of raters at diverse sites in multicenter trials presents a formidable challenge in calibration. Videoconferencing allows for the evaluation of IRR of raters at diverse sites by enabling raters at different sites to each independently interview a common patient. This is a more rigorous test of IRR than passive rating of videotapes. To evaluate the potential impact of videoconferencing on IRR, we compared IRR obtained via videoconference to IRR obtained using face-to-face interviews. Four raters at three different locations were paired using all pair-wise combinations of raters. Using videoconferencing, each paired rater independently conducted an interview with the same patient, who was at a third, central location. Raters were blind to each others' scores. ICC from this cohort (n=22) was not significantly different from the ICC obtained by a cohort using two face-to-face interviews (n=21) (0.90 vs. 0.93, respectively) nor from a cohort using one face-to-face interview and one remote interview (n=21) (0.88). The mean Hamilton Depression Rating Scale (HAMD) scores obtained were not significantly different. There appears to be no loss of signal using remote methods of calibration compared with traditional face-to-face methods.
Kobak
Kenneth A
KA
MedAvante Inc., 7601 Ganser Way, Madison, WI 53719, United States. kkobak@medavante.net
Williams
Janet B W
JB
Engelhardt
Nina
N
D003865:Major Depressive Disorder / Q000175:diagnosis* / Q000453:epidemiology / Q000523:psychology*; D006801:Humans; D015588:Observer Variation; D011795:Surveys and Questionnaires; D049269:Videoconferencing / Q000706:statistics & numerical data*