title: Observer agreement of the Manchester Triage System and the Emergency Severity Index: a simulation study.
abstract: OBJECTIVE
  To compare inter and intra-observer agreement of the Manchester Triage System (MTS) and the Emergency Severity Index (ESI).
  
  
  METHODS
  50 representative emergency department (ED) scenarios derived from actual cases were presented to 18 ED nurses from three different hospitals. Eight of them were familiar with MTS, six with ESI and four were not familiar but trained in both systems. They independently assigned triage scores to each scenario according to the triage system(s) they were familiar with. After 4-6 weeks the same nurses again judged the scenarios in a different order. Unanimity in judgement and unweighted and quadratic-weighted kappas were calculated.
  
  
  RESULTS
  Unanimity in judgement for MTS was 90% and for ESI 73%. One-level disagreement was found in 8% and 23% of the cases, respectively. Interobserver unweighted kappas were 0.76 (95% CI 0.68 to 0.83) for MTS and 0.46 (95% CI 0.37 to 0.55) for ESI. Quadratic-weighted kappas were 0.82 (95% CI 0.74 to 0.89) and 0.73 (95% CI 0.64 to 0.83), respectively. At 4-6 weeks, one-level intra-observer disagreements were 10% and 22% and 2-level disagreement 1% and 2%, respectively. Intra-observer unweighted kappas were 0.84 (95% CI 0.73 to 0.94) for MTS and 0.65 (95% CI 0.59 to 0.72) for ESI.
  
  
  CONCLUSIONS
  Using paper-based clinical scenarios, MTS was found to have a greater inter and intra-observer agreement than ESI.
authors.affiliation: Department of Emergency Medicine, Academic Medical Center, Amsterdam, The Netherlands. m.n.storm@amc.uva.nl
authors.forename: M N
authors.initials: MN
authors.lastname: Storm-Versloot
authors.forename: D T
authors.initials: DT
authors.lastname: Ubbink
authors.forename: V
authors.initials: V
authors.lastname: Chin a Choi
authors.forename: J S K
authors.initials: JS
authors.lastname: Luitse
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards; D004636:Emergency Service, Hospital / Q000706:statistics & numerical data; D006784:Hospitals, Teaching; D006786:Hospitals, Urban; D006801:Humans; D007600:Judgment; D009426:Netherlands; D009741:Nursing Staff, Hospital / Q000592:standards / Q000706:statistics & numerical data; D015588:Observer Variation; D010343:Patient Admission / Q000706:statistics & numerical data; D012720:Severity of Illness Index; D014218:Triage / Q000592:standards*