Evaluating helicopter emergency medical missions: a reliability study of the HEMS benefit and NACA scores.
BACKGROUND
The benefits of the Helicopter Emergency Medical Service (HEMS) and dispatch accuracy are continuously debated, and a widely accepted score to measure the benefits of the mission is lacking. The HEMS Benefit Score (HBS) has been used in Finnish helicopter emergency medical services, but studies are lacking. The National Advisory Committee for Aeronautics (NACA) score is widely used to measure the severity of illness or injury in the pre-hospital setting, but it has many critics due to its subjectivity. We investigated the inter-rater and rater-against-reference reliability of these scores.


METHODS
Twenty-five fictional HEMS missions were created by an expert panel. A total of 22 pre-hospital physicians were recruited to participate in the study from two different HEMS bases. The participants received written instructions on the use of the scores. Intraclass correlation coefficients (ICCs) and mean differences between rater-against-reference values were calculated.


RESULTS
A total of 17 physicians participated in the study. The ICC was 0.70 (95% CI 0.57-0.83) for the HBS and 0.65 (95% CI 0.51-0.79) for the NACA score. Mean differences between references and raters were -0.09 (SD 0.72) for the HBS and 0.28 (SD 0.61) for the NACA score, indicating that raters scored some lower NACA values than reference values formed by an expert panel.


CONCLUSIONS
The HBS and NACA score had substantial inter-rater reliability. In addition, the rater-against-reference values were acceptable, though large differences were observed between individual raters and references in some clinical cases.
Centre for Pre-Hospital Emergency Medicine, Oulu University Hospital, Oulu, Finland.|Medical Research Center, Research Group of Surgery, Anesthesiology and Intensive Care, Oulu University, Oulu, Finland.
L
http://orcid.org/0000-0001-5537-9749
L
Raatiniemi
Medical Research Center, Research Group of Surgery, Anesthesiology and Intensive Care, Oulu University, Oulu, Finland.|Oulu University Hospital, Department of Anesthesiology, Division of Intensive Care Medicine, Oulu, Finland.
J
J
Liisanantti
Emergency Medical Services, Turku University Hospital and University of Turku, Turku, Finland.
M
M
Tommila
Faculty of Medicine, Oulu University, Oulu, Finland.
S
S
Moilanen
Division of Operative Care, Oulu University Hospital, Oulu, Finland.
P
P
Ohtonen
Centre for Pre-Hospital Emergency Medicine, Oulu University Hospital, Oulu, Finland.|Medical Research Center, Research Group of Surgery, Anesthesiology and Intensive Care, Oulu University, Oulu, Finland.
M
M
Martikainen
Centre for Pre-Hospital Emergency Medicine, Oulu University Hospital, Oulu, Finland.
V
V
Voipio
Department of Anaesthesiology and Intensive Care Medicine, University of Helsinki and Helsinki University Central Hospital, Helsinki, Finland.
J
J
Reitala
Emergency Medical Services, Turku University Hospital and University of Turku, Turku, Finland.
T
T
Iirola
D026683:Advisory Committees; D017732:Air Ambulances / Q000706:statistics & numerical data*; D004632:Emergency Medical Services / Q000379:methods* / Q000706:statistics & numerical data*; D005387:Finland; D006801:Humans; D015203:Reproducibility of Results; D012720:Severity of Illness Index