Interrater reliability in clinical trials of depressive disorders.
OBJECTIVE
The goal of this study was to review the reported methods of rater training, assessment of interrater reliability, and rater drift in clinical trials of treatments for depressive disorders.


METHODS
Two psychiatrists independently identified all original reports of clinical trials relevant to depressive disorders published between 1996 and 2000 in the American Journal of Psychiatry and the Archives of General Psychiatry. Reported methods of rater training, assessment of interrater reliability, and rater drift were systematically summarized.


RESULTS
Sixty-three original papers met criteria for inclusion. Only 11 (17%) of the studies reported the number of raters. Only two (9%) of the 22 multicenter and four (10%) of the 41 single-center trials documented rater training. Only nine (22%) of the single-center trials and three (14%) of the multicenter trials reported interrater reliability, despite a median number of five raters (range=2-20). Only three (5%) of the 63 articles reported rater drift.


CONCLUSIONS
Few published reports of clinical trials of treatments for depressive disorders document adequately the number of raters, rater training, assessment of interrater reliability, and rater drift.
Mulsant
Benoit H
BH
Intervention Research Center for the Study of Late-Life Mood Disorders, Department of Biostatistics, University of Pittsburgh Health Care System, PA, USA. mulsantbh@msx.upmc.edu
Kastango
Kari B
KB
Rosen
Jules
J
Stone
Roslyn A
RA
Mazumdar
Sati
S
Pollock
Bruce G
BG
D003866:Depressive Disorder / Q000188:drug therapy / Q000628:therapy*; D006801:Humans; D015337:Multicenter Studies as Topic / Q000592:standards; D015588:Observer Variation; D010380:Peer Review / Q000379:methods / Q000592:standards; D010506:Periodicals as Topic / Q000592:standards; D011361:Professional Competence; D011594:Psychometrics / Q000592:standards; D016032:Randomized Controlled Trials as Topic / Q000379:methods / Q000592:standards*; D015203:Reproducibility of Results; D012108:Research Personnel / Q000193:education / Q000592:standards; D013223:Statistics as Topic / Q000592:standards; D016896:Treatment Outcome