title: The inter-rater agreement of retrospective assessments of adverse events does not improve with two reviewers per patient record.
abstract: OBJECTIVE
  To evaluate the inter-rater agreement of the record review process of the Dutch Adverse Event study, which we aimed to improve by the involvement of two independent physician reviewers per record instead of one including a consensus procedure in case of disagreement.
  
  
  METHODS
  The inter-rater agreement within pairs of physicians (independent review between physician A+B) and between pairs of physicians (independent review between physician A+B and C+D) was measured to evaluate the record review process with two physicians including a consensus procedure, with 4,272 and 119 records, respectively.
  
  
  RESULTS
  The inter-rater agreement within pairs of physicians was substantial for the determination of adverse events (AEs) with a kappa of 0.64 (95% confidence interval [CI]: 0.61, 0.68). The inter-rater agreement between pairs of physicians was fair for the determination of AEs with a kappa of 0.25 (95% CI: 0.05, 0.45).
  
  
  CONCLUSIONS
  A record review process with two physicians per record including a consensus procedure to assess AEs is not more reliable than a record review process with one physician. Retrospective estimates of incidence of AEs from record review studies should be interpreted with caution. Improvement of the method is necessary for monitoring incidence of AEs over time at a national level.
authors.affiliation: NIVEL, Netherlands Institute for Health Services Research, P.O. box 1568, 3500 BN Utrecht, The Netherlands. m.zegers@nivel.nl
authors.forename: Marieke
authors.initials: M
authors.lastname: Zegers
authors.forename: Martine C
authors.initials: MC
authors.lastname: de Bruijne
authors.forename: Cordula
authors.initials: C
authors.lastname: Wagner
authors.forename: Peter P
authors.initials: PP
authors.lastname: Groenewegen
authors.forename: Gerrit
authors.initials: G
authors.lastname: van der Wal
authors.forename: Henrica C W
authors.initials: HC
authors.lastname: de Vet
mesh_terms: D032921:Consensus; D006760:Hospitalization; D006801:Humans; D019300:Medical Errors / Q000517:prevention & control / Q000706:statistics & numerical data*; D008499:Medical Records / Q000592:standards*; D009426:Netherlands; D015588:Observer Variation; D015203:Reproducibility of Results; D017751:Safety Management / Q000379:methods*