title: Metrics of success: Measuring impact of a departmental near-miss incident learning system.
abstract: OBJECTIVE
  There is a growing interest in the application of incident learning systems (ILS) to radiation oncology. The purpose of the present study is to define statistical metrics that may serve as benchmarks for successful operation of an incident learning system.
  
  
  METHODS
  A departmental safety and quality ILS was developed to monitor errors, near-miss events, and process improvement suggestions. Event reports were reviewed by a multiprofessional quality improvement committee. Events were scored by a near-miss risk index (NMRI) and categorized by event point of origination and discovery. Reporting trends were analyzed over a 2-year period, including total number and rates of events reported, users reporting, NMRI, and event origination and discovery.
  
  
  RESULTS
  A total of 1897 reports were evaluated (1.0 reports/patient, 0.9 reports/unique treatment course). Participation in the ILS increased as demonstrated by total events (2.1 additional reports/month) and unique users (0.5 new users/month). Sixteen percent of reports had an NMRI of 0 (none), 42% had an NMRI of 1 (mild), 25% had an NMRI of 2 (moderate), 12% had an NMRI of 3 (severe), and 5% had an NMRI of 4 (critical). Event NMRI showed a significant decrease in the first 6 months (1.68-1.42, P < .001). Trends in origination and discovery of reports were broadly distributed between radiation therapy process steps and staff groups. The highest risk events originated in imaging for treatment planning (NMRI = 2.0 ± 1.1; P < .0001) and were detected in on-treatment quality management (NMRI = 1.7 ± 1.1; P = .003).
  
  
  CONCLUSIONS
  Over the initial 2-year period of ILS operation, rates of reporting increased, staff participation increased, and NMRI of reported events declined. These data mirror previously reported findings of improvement in safety culture endpoints. These metrics may be useful for other institutions seeking to create or evaluate their own ILS.
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington. Electronic address: nyflot@uw.edu.
authors.forename: Matthew J
authors.initials: MJ
authors.lastname: Nyflot
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Jing
authors.initials: J
authors.lastname: Zeng
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Aaron S
authors.initials: AS
authors.lastname: Kusano
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Avrey
authors.initials: A
authors.lastname: Novak
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Thomas D
authors.initials: TD
authors.lastname: Mullen
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Wendy
authors.initials: W
authors.lastname: Gao
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Loucille
authors.initials: L
authors.lastname: Jordan
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Patricia A
authors.initials: PA
authors.lastname: Sponseller
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Joshua C
authors.initials: JC
authors.lastname: Carlson
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Gabrielle
authors.initials: G
authors.lastname: Kane
authors.affiliation: Department of Radiation Oncology, University of Washington, Seattle, Washington.
authors.forename: Eric C
authors.initials: EC
authors.lastname: Ford
mesh_terms: D032921:Consensus; D006801:Humans; D061214:Patient Safety / Q000592:standards*; D018794:Problem-Based Learning / Q000379:methods*; D058996:Quality Improvement; D018787:Radiation Oncology / Q000592:standards*; D012308:Risk Management / Q000379:methods* / Q000592:standards*