title: Can medical record reviewers reliably identify errors and adverse events in the ED?
abstract: BACKGROUND
  Chart review has been the mainstay of medical quality assurance practices since its introduction more than a century ago. The validity of chart review, however, has been vitiated by a lack of methodological rigor.
  
  
  OBJECTIVE
  By measuring the degree of interrater agreement among a 13-member review board of emergency physicians, we sought to validate the reliability of a chart review-based quality assurance process using computerized screening based on explicit case parameters.
  
  
  METHODS
  All patients presenting to an urban, tertiary care academic medical center emergency department (annual volume of 57,000 patients) between November 2012 and November 2013 were screened electronically. Cases were programmatically flagged for review according to explicit criteria: return within 72hours, procedural evaluation, floor-to-ICU transfer within 24hours of admission, death within 24hours of admission, physician complaints, and patient complaints. Each case was reviewed independently by a 13-member emergency department quality assurance committee all of whom were board certified in emergency medicine and trained in the use of the tool. None of the reviewers were involved in the care of the specific patients reviewed by them. Reviewers used a previously validated 8-point Likert scale to rate the (1) coordination of patient care, (2) presence and severity of adverse events, (3) degree of medical error, and (4) quality of medical judgment. Agreement among reviewers was assessed with the intraclass correlation coefficient (ICC) for each parameter.
  
  
  RESULTS
  Agreement and the degree of significance for each parameter were as follows: coordination of patient care (ICC=0.67; P<.001), presence and severity of adverse events (ICC=0.52; P=.001), degree of medical error (ICC=0.72; P<.001), and quality of medical judgment (ICC=0.67; P<.001).
  
  
  CONCLUSIONS
  Agreement in the chart review process can be achieved among physician-reviewers. The degree of agreement attainable is comparable to or superior to that of similar studies reported to date. These results highlight the potential for the use of computerized screening, explicit criteria, and training of expert reviewers to improve the reliability and validity of chart review-based quality assurance.
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine. Electronic address: rklasco@bidmc.harvard.edu.
authors.forename: Richard S
authors.initials: RS
authors.lastname: Klasco
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Richard E
authors.initials: RE
authors.lastname: Wolfe
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Terrance
authors.initials: T
authors.lastname: Lee
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Philip
authors.initials: P
authors.lastname: Anderson
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Lee S
authors.initials: LS
authors.lastname: Jacobson
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Joshua
authors.initials: J
authors.lastname: Solano
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Edlow
authors.affiliation: Beth Israel Deaconess Medical Center, Department of Emergency Medicine.
authors.forename: Shamai A
authors.initials: SA
authors.lastname: Grossman
mesh_terms: D015331:Cohort Studies; D004636:Emergency Service, Hospital; D006801:Humans; D019300:Medical Errors; D008499:Medical Records; D015588:Observer Variation; D011785:Quality Assurance, Health Care; D015203:Reproducibility of Results