Failure mode and effects analysis outputs: are they valid?
BACKGROUND
Failure Mode and Effects Analysis (FMEA) is a prospective risk assessment tool that has been widely used within the aerospace and automotive industries and has been utilised within healthcare since the early 1990s. The aim of this study was to explore the validity of FMEA outputs within a hospital setting in the United Kingdom.


METHODS
Two multidisciplinary teams each conducted an FMEA for the use of vancomycin and gentamicin. Four different validity tests were conducted: Face validity: by comparing the FMEA participants' mapped processes with observational work. Content validity: by presenting the FMEA findings to other healthcare professionals. Criterion validity: by comparing the FMEA findings with data reported on the trust's incident report database. Construct validity: by exploring the relevant mathematical theories involved in calculating the FMEA risk priority number.


RESULTS
Face validity was positive as the researcher documented the same processes of care as mapped by the FMEA participants. However, other healthcare professionals identified potential failures missed by the FMEA teams. Furthermore, the FMEA groups failed to include failures related to omitted doses; yet these were the failures most commonly reported in the trust's incident database. Calculating the RPN by multiplying severity, probability and detectability scores was deemed invalid because it is based on calculations that breach the mathematical properties of the scales used.


CONCLUSIONS
There are significant methodological challenges in validating FMEA. It is a useful tool to aid multidisciplinary groups in mapping and understanding a process of care; however, the results of our study cast doubt on its validity. FMEA teams are likely to need different sources of information, besides their personal experience and knowledge, to identify potential failures. As for FMEA's methodology for scoring failures, there were discrepancies between the teams' estimates and similar incidents reported on the trust's incident database. Furthermore, the concept of multiplying ordinal scales to prioritise failures is mathematically flawed. Until FMEA's validity is further explored, healthcare organisations should not solely depend on their FMEA results to prioritise patient safety issues.
Department of Practice and Policy, UCL School of Pharmacy, BMA House, Mezzanine Floor, Tavistock Square, London, WC1H 9JP, UK.
Nada Atef
NA
Shebl
Bryony Dean
BD
Franklin
Nick
N
Barber
D000900:Anti-Bacterial Agents / Q000008:administration & dosage*; D019091:Critical Pathways; D005839:Gentamicins / Q000008:administration & dosage; D007722:Health Knowledge, Attitudes, Practice; D006784:Hospitals, Teaching; D006801:Humans; D007262:Infusions, Intravenous / Q000379:methods; D008508:Medication Errors / Q000517:prevention & control; D009313:National Health Programs; D010348:Patient Care Team / Q000592:standards; D036301:Qualitative Research; D012008:Recurrence; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000592:standards*; D012308:Risk Management; D013997:Time Factors; D017211:Treatment Failure; D006113:United Kingdom; D014640:Vancomycin / Q000008:administration & dosage
D000900:Anti-Bacterial Agents; D005839:Gentamicins; D014640:Vancomycin