Analysis of unintended events in hospitals: inter-rater reliability of constructing causal trees and classifying root causes.
BACKGROUND
Root cause analysis is a method to examine causes of unintended events. PRISMA (Prevention and Recovery Information System for Monitoring and Analysis: is a root cause analysis tool. With PRISMA, events are described in causal trees and root causes are subsequently classified with the Eindhoven Classification Model (ECM). It is important that root cause analysis tools are reliable, because they form the basis for patient safety interventions.


OBJECTIVE
Determining the inter-rater reliability of descriptions, number and classifications of root causes.


METHODS
Totally, 300 unintended event reports were sampled from a database of 2028 events in 30 hospital units. The reports were previously analysed using PRISMA by experienced analysts and were re-analysed to compare descriptions and number of root causes (n = 150) and to determine the inter-rater reliability of classifications (n = 150).


METHODS
Percentage agreement and Cohen's kappa (kappa).


RESULTS
Agreement between descriptions of root causes was satisfactory: 54% agreement, 17% partial agreement and 29% no agreement. Inter-rater reliability of number of root causes was moderate (kappa = 0.46). Inter-rater reliability of classifying root causes with the ECM was substantial from highest category level (kappa = 0.71) to lowest subcategory level (kappa = 0.63). Most discrepancies occurred in classifying external causes.


CONCLUSIONS
Results indicate that causal tree analysis with PRISMA is reliable. Analysts formulated similar root causes and agreed considerably on classifications, but showed variation in number of root causes. More training on disclosure of all relevant root causes is recommended as well as adjustment of the model by combining all external causes into one category.
Smits
Marleen
M
Netherlands Institute for Health Services Research, PO Box 1568, NIVEL, 3500 BN Utrecht, The Netherlands. m.smits@nivel.nl
Janssen
Jasper
J
de Vet
Riekie
R
Zwaan
Laura
L
Timmermans
Danielle
D
Groenewegen
Peter
P
Wagner
Cordula
C
D015984:Causality; D004282:Documentation; D006739:Hospital Administration; D006801:Humans; D019300:Medical Errors / Q000379:methods* / Q000517:prevention & control; D015588:Observer Variation; D011785:Quality Assurance, Health Care / Q000379:methods*; D019984:Quality Indicators, Health Care; D017751:Safety Management / Q000379:methods*