title: Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises.
abstract: BACKGROUND
  Exercises that simulate emergency scenarios are accepted widely as an essential component of a robust Emergency Preparedness program. Unfortunately, the variability in the quality of the exercises conducted, and the lack of standardized processes to measure performance, has limited the value of exercises in measuring preparedness.
  
  
  METHODS
  In order to help health organizations improve the quality and standardization of the performance data they collect during simulated emergencies, a model online exercise evaluation toolkit was developed using performance measures tested in over 60 Emergency Preparedness exercises. The exercise evaluation toolkit contains three major components: (1) a database of measures that can be used to assess performance during an emergency response exercise; (2) a standardized data collection tool (form); and (3) a program that populates the data collection tool with the measures that have been selected by the user from the database. The evaluation toolkit was pilot tested from January through September 2014 in collaboration with 14 partnering organizations representing 10 public health agencies and four health care agencies from eight states across the US. Exercise planners from the partnering organizations were asked to use the toolkit for their exercise evaluation process and were interviewed to provide feedback on the use of the toolkit, the generated evaluation tool, and the usefulness of the data being gathered for the development of the exercise after-action report.
  
  
  RESULTS
  Ninety-three percent (93%) of exercise planners reported that they found the online database of performance measures appropriate for the creation of exercise evaluation forms, and they stated that they would use it again for future exercises. Seventy-two percent (72%) liked the exercise evaluation form that was generated from the toolkit, and 93% reported that the data collected by the use of the evaluation form were useful in gauging their organization's performance during the exercise. Seventy-nine percent (79%) of exercise planners preferred the evaluation form generated by the toolkit to other forms of evaluations.
  
  
  CONCLUSIONS
  Results of this project show that users found the newly developed toolkit to be user friendly and more relevant to measurement of specific public health and health care capabilities than other tools currently available. The developed toolkit may contribute to the further advancement of developing a valid approach to exercise performance measurement.
authors.affiliation: 1Division of Policy Translation and Leadership Development,Harvard T.H. Chan School of Public Health,Boston,MassachusettsUSA.
authors.forename: Foluso
authors.initials: F
authors.lastname: Agboola
authors.affiliation: 1Division of Policy Translation and Leadership Development,Harvard T.H. Chan School of Public Health,Boston,MassachusettsUSA.
authors.forename: Dorothy
authors.initials: D
authors.lastname: Bernard
authors.affiliation: 1Division of Policy Translation and Leadership Development,Harvard T.H. Chan School of Public Health,Boston,MassachusettsUSA.
authors.forename: Elena
authors.initials: E
authors.lastname: Savoia
authors.affiliation: 1Division of Policy Translation and Leadership Development,Harvard T.H. Chan School of Public Health,Boston,MassachusettsUSA.
authors.forename: Paul D
authors.initials: PD
authors.lastname: Biddinger
mesh_terms: D002959:Civil Defense / Q000592:standards*; D004189:Disaster Planning; D060731:Emergency Responders; D006801:Humans; D011361:Professional Competence / Q000592:standards*; D015397:Program Evaluation / Q000379:methods*; D012984:Software
keywords: ASPR Assistant Secretary for Preparedness and Response; CDC Centers for Disease Control and Prevention; EEG exercise evaluation guides; HPP hospital preparedness capability; HSEEP Homeland Security Exercise Evaluation Program; PHEP public health preparedness capability; disaster planning; emergency preparedness exercise; exercise evaluation; measures; performance improvement