title: Widely used track and trigger scores: are they ready for automation in practice?
abstract: BACKGROUND
  Early Warning Scores (EWS) are widely used for early recognition of patient deterioration. Automated alarm/alerts have been recommended as a desirable characteristic for detection systems of patient deterioration. We undertook a comparative analysis of performance characteristics of common EWS methods to assess how they would function if automated.
  
  
  METHODS
  We evaluated the most widely used EWS systems (MEWS, SEWS, GMEWS, Worthing, ViEWS and NEWS) and the Rapid Response Team (RRT) activation criteria in use in our institution. We compared their ability to predict the composite outcome of Resuscitation call, RRS activation or unplanned transfer to the ICU, in a time-dependent manner (3, 8, 12, 24 and 36 h after the observation) by determining the sensitivity, specificity and positive predictive values (PPV). We used a large vital signs database (6,948,689 unique time points) from 34,898 unique consecutive hospitalized patients.
  
  
  RESULTS
  PPVs ranged from less than 0.01 (Worthing, 3 h) to 0.21 (GMEWS, 36 h). Sensitivity ranged from 0.07 (GMEWS, 3 h) to 0.75 (ViEWS, 36 h). Used in an automated fashion, these would correspond to 1040-215,020 false positive alerts per year.
  
  
  CONCLUSIONS
  When the evaluation is performed in a time-sensitive manner, the most widely used weighted track-and-trigger scores do not offer good predictive capabilities for use as criteria for an automated alarm system. For the implementation of an automated alarm system, better criteria need to be developed and validated before implementation.
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Santiago
authors.initials: S
authors.lastname: Romero-Brufau
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States; Division of Hospital Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States; Division of Health Care Policy and Research, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States. Electronic address: huddleston.jeanne@mayo.edu.
authors.forename: Jeanne M
authors.initials: JM
authors.lastname: Huddleston
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States; Division of Health Care Policy and Research, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: James M
authors.initials: JM
authors.lastname: Naessens
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States; Division of Health Care Policy and Research, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Matthew G
authors.initials: MG
authors.lastname: Johnson
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States; Division of Health Care Policy and Research, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Joel
authors.initials: J
authors.lastname: Hickman
authors.affiliation: Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN 55905, United States; Division of Health Care Policy and Research, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Bruce W
authors.initials: BW
authors.lastname: Morlan
authors.affiliation: Department of Anesthesiology, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Jeffrey B
authors.initials: JB
authors.lastname: Jensen
authors.affiliation: Division of Pulmonary Medicine and Critical Care, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Sean M
authors.initials: SM
authors.lastname: Caples
authors.affiliation: Department of Nursing, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Jennifer L
authors.initials: JL
authors.lastname: Elmer
authors.affiliation: Department of Nursing, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Julie A
authors.initials: JA
authors.lastname: Schmidt
authors.affiliation: Division of Pulmonary Medicine and Critical Care, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Timothy I
authors.initials: TI
authors.lastname: Morgenthaler
authors.affiliation: Department of Laboratory Medicine and Pathology, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, United States.
authors.forename: Paula J
authors.initials: PJ
authors.lastname: Santrach
mesh_terms: D000368:Aged; D003422:Critical Care; D020000:Decision Support Systems, Clinical; D005240:Feasibility Studies; D005260:Female; D006305:Health Status Indicators; D057209:Hospital Rapid Response Team; D006760:Hospitalization; D006801:Humans; D008297:Male; D050316:Medical Order Entry Systems; D008875:Middle Aged; D010355:Patient Identification Systems; D011237:Predictive Value of Tests; D012151:Resuscitation; D012189:Retrospective Studies; D018570:Risk Assessment; D013997:Time Factors; D055986:Vital Signs
keywords: Automation; Cardiorespiratory arrests; Early warning score; Patient safety; Rapid response system; Track-and-trigger