title: Validating FMEA output against incident learning data: A study in stereotactic body radiation therapy.
abstract: OBJECTIVE
  Though failure mode and effects analysis (FMEA) is becoming more widely adopted for risk assessment in radiation therapy, to our knowledge, its output has never been validated against data on errors that actually occur. The objective of this study was to perform FMEA of a stereotactic body radiation therapy (SBRT) treatment planning process and validate the results against data recorded within an incident learning system.
  
  
  METHODS
  FMEA on the SBRT treatment planning process was carried out by a multidisciplinary group including radiation oncologists, medical physicists, dosimetrists, and IT technologists. Potential failure modes were identified through a systematic review of the process map. Failure modes were rated for severity, occurrence, and detectability on a scale of one to ten and risk priority number (RPN) was computed. Failure modes were then compared with historical reports identified as relevant to SBRT planning within a departmental incident learning system that has been active for two and a half years. Differences between FMEA anticipated failure modes and existing incidents were identified.
  
  
  RESULTS
  FMEA identified 63 failure modes. RPN values for the top 25% of failure modes ranged from 60 to 336. Analysis of the incident learning database identified 33 reported near-miss events related to SBRT planning. Combining both methods yielded a total of 76 possible process failures, of which 13 (17%) were missed by FMEA while 43 (57%) identified by FMEA only. When scored for RPN, the 13 events missed by FMEA ranked within the lower half of all failure modes and exhibited significantly lower severity relative to those identified by FMEA (p = 0.02).
  
  
  CONCLUSIONS
  FMEA, though valuable, is subject to certain limitations. In this study, FMEA failed to identify 17% of actual failure modes, though these were of lower risk. Similarly, an incident learning system alone fails to identify a large number of potentially high-severity process errors. Using FMEA in combination with incident learning may render an improved overview of risks within a process.
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: F
authors.initials: F
authors.lastname: Yang
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: N
authors.initials: N
authors.lastname: Cao
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: L
authors.initials: L
authors.lastname: Young
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: J
authors.initials: J
authors.lastname: Howard
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: W
authors.initials: W
authors.lastname: Logan
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: T
authors.initials: T
authors.lastname: Arbuckle
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: P
authors.initials: P
authors.lastname: Sponseller
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: T
authors.initials: T
authors.lastname: Korssjoen
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: J
authors.initials: J
authors.lastname: Meyer
authors.affiliation: Department of Radiation Oncology, University of Washington Medical Center, Seattle, Washington 98195.
authors.forename: E
authors.initials: E
authors.lastname: Ford
mesh_terms: D016208:Databases, Factual; D019544:Equipment Failure Analysis; D006801:Humans; D000069550:Machine Learning; D016634:Radiosurgery / Q000379:methods*; D011880:Radiotherapy Planning, Computer-Assisted / Q000379:methods*; D018570:Risk Assessment