title: Evaluating implementation of the emergency severity index in a Belgian hospital.
abstract: BACKGROUND
  Triage aims to categorize patients based on their clinical need and the available departmental resources. To accomplish this goal, one needs to ensure that the implemented triage system is reliable and that staff use it correctly. Therefore this study assessed the ability of Belgium nurses to apply the Emergency Severity Index (ESI), version 4, to hypothetical case scenarios after an educational intervention.
  
  
  METHODS
  An ESI educational intervention was implemented in accordance with the ESI manual. Using paper case scenarios, nurses' interrater agreement was assessed by comparing triage nurse ESI levels with the reference answers noted in the implementation manual. Interrater agreement was measured by the percentage of agreement and Cohen's κ coefficient using different weighting schemes.
  
  
  RESULTS
  Overall, 77.5% of the scenario cases were coded according the ESI guidelines, resulting in a good interrater agreement (κ = 0.72, linear weighted κ = 0.84, quadratic weighted κ = 0.92, and triage-weighted scheme = 0.79). Interrater agreement varied when evaluating each ESI level separately. Undertriage was more common than overtriage. The highest misclassification range (37.8%) occurred in ESI level 2 scenarios, with 99.2% of the misclassifications being undertriaged.
  
  
  CONCLUSIONS
  Implementation of the ESI into a novel setting guided by a locally developed training program resulted in suboptimal interrater agreement. Existing weighted κ schemes overestimated the interrater agreement between the triage nurse-assigned ESI level and the reference standard. By providing an aggregated measure of agreement, which allows partial agreement, clinically significant misclassification was masked by a misleading "good" interrater agreement.
authors.affiliation: Hasselt, Leuven, and Ghent, Belgium. Electronic address: jochen.bergs@uhasselt.be.
authors.forename: Jochen
authors.initials: J
authors.lastname: Bergs
authors.affiliation: Hasselt, Leuven, and Ghent, Belgium.
authors.forename: Sandra
authors.initials: S
authors.lastname: Verelst
authors.affiliation: Hasselt, Leuven, and Ghent, Belgium.
authors.forename: Jean-Bernard
authors.initials: JB
authors.lastname: Gillet
authors.affiliation: Hasselt, Leuven, and Ghent, Belgium.
authors.forename: Dominique
authors.initials: D
authors.lastname: Vandijck
mesh_terms: D001530:Belgium; D003430:Cross-Sectional Studies; D016529:Emergency Nursing / Q000193:education*; D004636:Emergency Service, Hospital / Q000592:standards*; D006801:Humans; D009730:Nursing Assessment; D015397:Program Evaluation; D012720:Severity of Illness Index; D014218:Triage / Q000379:methods*
keywords: Emergency nursing; Emergency service, hospital; Implementation; Triage