title: The reliability of AHRQ Common Format Harm Scales in rating patient safety events.
abstract: OBJECTIVE
  A study was conducted to determine the reliability of Agency for Healthcare Research & Quality (AHRQ) Common Format Harm Scale versions 1.1 and 1.2 in rating patient safety events among users of the UHC Patient Safety Net, a Web-based incident reporting tool.
  
  
  METHODS
  To test interrater agreement, UHC developed a survey tool consisting of patient event scenarios. In 2011, a survey evaluating Harm Scale v.1.1 was distributed to 921 quality, risk, and safety (QRS) managers at 89 organizations; in 2012, a second survey evaluating Harm Scale v.1.2 was sent to 13,280 managers at 102 organizations.
  
  
  RESULTS
  Regardless of the version used, in 3 of 9 scenarios, fewer than 60% of respondents agreed on a single score. Interrater agreement increased for certain event scenarios with v.1.2 but decreased for other scenarios. Interrater reliability was moderate for both v.1.1 (k = 0.51) and v.1.2 (k = 0.47). Interrater agreement improved in v.1.2 when results were limited to more experienced raters but still remained in the moderate range (k = 0.58).
  
  
  CONCLUSIONS
  AHRQ Common Format Harm Scale v.1.1 and v.1.2 both had moderate interrater reliability. Using Harm Scale v.1.1, respondents had difficulty distinguishing "injury limited to additional treatment" from "temporary harm," whereas, using Harm Scale v.1.2, respondents had difficulty distinguishing moderate harm from one of the adjacent levels-mild or severe harm. This study provides valuable data that can inform harm scale revision to improve the quality of aggregate safety data used to define and direct safety efforts.
authors.affiliation: From the *Project Manager, Patient Safety Research, †Director of Quality Research, ‡Director of Patent Safety, §Senior Patient Safety Consultant, and ∥Vice President of Performance Improvement, UHC, Chicago, Illinois.
authors.forename: Tamara
authors.initials: T
authors.lastname: Williams
authors.forename: Marilyn
authors.initials: M
authors.lastname: Szekendi
authors.forename: Stephen
authors.initials: S
authors.lastname: Pavkovic
authors.forename: Wanda
authors.initials: W
authors.lastname: Clevenger
authors.forename: Julie
authors.initials: J
authors.lastname: Cerese
mesh_terms: D003625:Data Collection; D006801:Humans; D061214:Patient Safety; D015203:Reproducibility of Results; D012308:Risk Management; D017751:Safety Management; D014481:United States; D016270:United States Agency for Healthcare Research and Quality