title: Inter- and intrarater reliability of retrospective drug utilization reviewers.
abstract: OBJECTIVE
  To assess inter- and intrarater reliability among 23 pharmacist and physician retrospective drug utilization reviewers and to assess interrater reliability after a reviewer training session.
  
  
  METHODS
  Exploratory study.
  
  
  METHODS
  Maryland Medicaid's retrospective drug utilization review (DUR) program.
  
  
  METHODS
  23 physician and pharmacist retrospective drug utilization reviewers.
  
  
  METHODS
  None.
  
  
  METHODS
  Profiles rated as "intervention indicated" or "intervention not indicated." Cochran's Q test, overall percent agreement, and the unweighted kappa statistic were used in the analysis of review consistency.
  
  
  RESULTS
  Intrarater reliability showed substantial consistency among the 23 reviewers; the percent agreement was 82.9% with kappa = 0.66. Interrater reliability, however, was poor, with an overall agreement of 69.6% and kappa = 0.16. Interrater reliability was also poor after a one-hour reviewer training session (agreement 81.8%, kappa = -0.19).
  
  
  CONCLUSIONS
  The implicit review process used in the retrospective DUR program that we evaluated was unreliable. Since reliability is a necessary but not sufficient condition for validity of an indicator of inappropriate drug use, the validity of the DUR implicit review process is in question.
authors.affiliation: School of Pharmacy, University of Maryland, Baltimore 21201, USA. zuckerma@pharmacy.ab.umd.edu
authors.forename: I H
authors.initials: IH
authors.lastname: Zuckerman
authors.forename: D M
authors.initials: DM
authors.lastname: Mulhearn
authors.forename: C J
authors.initials: CJ
authors.lastname: Metge
mesh_terms: D003627:Data Interpretation, Statistical; D017723:Drug Utilization Review / Q000458:organization & administration*; D006801:Humans; D008396:Maryland; D008484:Medicaid; D015588:Observer Variation; D010380:Peer Review; D015203:Reproducibility of Results; D012189:Retrospective Studies; D014481:United States