title: Assessing Reliability of Medical Record Reviews for the Detection of Hospital Adverse Events.
abstract: OBJECTIVE
  The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events.
  
  
  METHODS
  We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later.
  
  
  RESULTS
  In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89).
  
  
  CONCLUSIONS
  In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.
authors.affiliation: Department of Preventive Medicine, University of Ulsan College of Medicine, Seoul, Korea.
authors.forename: Minsu
authors.initials: M
authors.lastname: Ock
authors.affiliation: Department of Preventive Medicine, University of Ulsan College of Medicine, Seoul, Korea.
authors.forename: Sang-il
authors.initials: SI
authors.lastname: Lee
authors.affiliation: Department of Preventive Medicine, University of Ulsan College of Medicine, Seoul, Korea.
authors.forename: Min-Woo
authors.initials: MW
authors.lastname: Jo
authors.affiliation: Public Health Medical Service, Seoul National University Boramae Medical Center, Seoul, Korea.
authors.forename: Jin Yong
authors.initials: JY
authors.lastname: Lee
authors.affiliation: Department of Nursing, Dankook University, Cheonan, Korea.
authors.forename: Seon-Ha
authors.initials: SH
authors.lastname: Kim
mesh_terms: D064420:Drug-Related Side Effects and Adverse Reactions / Q000145:classification / Q000473:pathology*; D006769:Hospitals, General; D006801:Humans; D008499:Medical Records; D009726:Nurses / Q000523:psychology; D010820:Physicians / Q000523:psychology; D012189:Retrospective Studies
keywords: Adverse event; Inter-rater reliability; Intra-rater reliability; Medical record review; Patient safety