Inter- and intrarater reliability of retrospective drug utilization reviewers.
OBJECTIVE
To assess inter- and intrarater reliability among 23 pharmacist and physician retrospective drug utilization reviewers and to assess interrater reliability after a reviewer training session.


METHODS
Exploratory study.


METHODS
Maryland Medicaid's retrospective drug utilization review (DUR) program.


METHODS
23 physician and pharmacist retrospective drug utilization reviewers.


METHODS
None.


METHODS
Profiles rated as "intervention indicated" or "intervention not indicated." Cochran's Q test, overall percent agreement, and the unweighted kappa statistic were used in the analysis of review consistency.


RESULTS
Intrarater reliability showed substantial consistency among the 23 reviewers; the percent agreement was 82.9% with kappa = 0.66. Interrater reliability, however, was poor, with an overall agreement of 69.6% and kappa = 0.16. Interrater reliability was also poor after a one-hour reviewer training session (agreement 81.8%, kappa = -0.19).


CONCLUSIONS
The implicit review process used in the retrospective DUR program that we evaluated was unreliable. Since reliability is a necessary but not sufficient condition for validity of an indicator of inappropriate drug use, the validity of the DUR implicit review process is in question.
Zuckerman
I H
IH
School of Pharmacy, University of Maryland, Baltimore 21201, USA. zuckerma@pharmacy.ab.umd.edu
Mulhearn
D M
DM
Metge
C J
CJ
D003627:Data Interpretation, Statistical; D017723:Drug Utilization Review / Q000458:organization & administration*; D006801:Humans; D008396:Maryland; D008484:Medicaid; D015588:Observer Variation; D010380:Peer Review; D015203:Reproducibility of Results; D012189:Retrospective Studies; D014481:United States