title: Inter-agency communication and operations capabilities during a hospital functional exercise: reliability and validity of a measurement tool.
abstract: BACKGROUND
  As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise.
  
  
  METHODS
  The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.
  
  
  RESULTS
  Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self reported scores and external evaluators' scores ranged from moderate to good.
  
  
  CONCLUSIONS
  The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.
authors.affiliation: Center for Public Health Preparedness, Harvard School of Public Health, Boston, MA, USA. esavoia@hsph.harvard.edu
authors.forename: Elena
authors.initials: E
authors.lastname: Savoia
authors.forename: Paul D
authors.initials: PD
authors.lastname: Biddinger
authors.forename: Jon
authors.initials: J
authors.lastname: Burstein
authors.forename: Michael A
authors.initials: MA
authors.lastname: Stoto
mesh_terms: D003142:Communication; D004189:Disaster Planning / Q000458:organization & administration*; D006747:Hospital Communication Systems / Q000458:organization & administration*; D006801:Humans; D008404:Massachusetts; D017146:Models, Organizational; D011361:Professional Competence; D011446:Prospective Studies; D011634:Public Health; D015203:Reproducibility of Results; D013223:Statistics as Topic; D011795:Surveys and Questionnaires