title: Measuring Harm in Health Care: Optimizing Adverse Event Review.
abstract: The objective of this study was to identify modifiable factors that improve the reliability of ratings of severity of health care-associated harm in clinical practice improvement and research.
  
  
  
  A diverse group of clinicians rated 8 types of adverse events: blood product, device or medical/surgical supply, fall, health care-associated infection, medication, perinatal, pressure ulcer, surgery. We used a generalizability theory framework to estimate the impact of number of raters, rater experience, and rater provider type on reliability.
  
  
  
  Pharmacists were slightly more precise and consistent in their ratings than either physicians or nurses. For example, to achieve high reliability of 0.83, 3 physicians could be replaced by 2 pharmacists without loss in precision of measurement. If only 1 rater was available for rating, ∼5% of the reviews for severe harm would have been incorrectly categorized. Reliability was greatly improved with 2 reviewers.
  
  
  
  We identified factors that influence the reliability of clinician reviews of health care-associated harm. Our novel use of generalizability analyses improved our understanding of how differences affect reliability. This approach was useful in optimizing resource utilization when selecting raters to assess harm and may have similar applications in other settings in health care.
authors.affiliation: *James M. Anderson Center †Department of Pediatrics, Cincinnati Children's Hospital, Cincinnati, OH ‡National Board of Medical Examiners, Philadelphia, PA §Meyers Primary Care Institute, Worcester, MA ∥Agency for Healthcare Quality and Research, Rockville, MD ¶Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester #Kindred Healthcare, Wellesley, MA.
authors.forename: Kathleen E
authors.initials: KE
authors.lastname: Walsh
authors.forename: Polina
authors.initials: P
authors.lastname: Harik
authors.forename: Kathleen M
authors.initials: KM
authors.lastname: Mazor
authors.forename: Deborah
authors.initials: D
authors.lastname: Perfetto
authors.forename: Milena
authors.initials: M
authors.lastname: Anatchkova
authors.forename: Colleen
authors.initials: C
authors.lastname: Biggins
authors.forename: Joann
authors.initials: J
authors.lastname: Wagner
authors.forename: Pamela J
authors.initials: PJ
authors.lastname: Schoettker
authors.forename: Cassandra
authors.initials: C
authors.lastname: Firneno
authors.forename: Robert
authors.initials: R
authors.lastname: Klugman
authors.forename: Jennifer
authors.initials: J
authors.lastname: Tjia
mesh_terms: D001291:Attitude of Health Personnel; D064420:Drug-Related Side Effects and Adverse Reactions; D040261:Harm Reduction; D006801:Humans; D007049:Iatrogenic Disease; D019300:Medical Errors / Q000706:statistics & numerical data*; D010380:Peer Review; D011446:Prospective Studies; D015203:Reproducibility of Results; D014481:United States