Assessing public health capabilities during emergency preparedness tabletop exercises: reliability and validity of a measurement tool.
OBJECTIVE
Improving the ability of local public health agencies to respond to large-scale emergencies is an ongoing challenge. Tabletop exercises can provide an opportunity for individuals and groups to practice coordination of emergency response and evaluate performance. The purpose of this study was to develop a valid and reliable self-assessment performance measurement tool for tabletop exercise participants.


METHODS
The study population comprised 179 public officials who attended three tabletop exercises in Massachusetts and Maine between September 2005 and November 2006. A 42-item questionnaire was developed to assess five public health functional capabilities: (1) leadership and management, (2) mass casualty care, (3) communication, (4) disease control and prevention, and (5) surveillance and epidemiology. Analyses were undertaken to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.


RESULTS
Thirty-seven questions were retained in the final questionnaire and grouped according to the original five domains. Alpha coefficients were 0.81 or higher for all scales. The five-factor solution from the principal components analysis accounted for 60% of the total variance, and the factor structure was consistent with the five domains of the original conceptual model. Inter-rater agreement ranged from good to excellent.


CONCLUSIONS
The resulting 37-item performance measurement tool was found to reliably measure public health functional capabilities in a tabletop exercise setting, with preliminary evidence of a factor structure consistent with the original conceptualization and of criterion-related validity.
Savoia
Elena
E
Center for Public Health Preparedness, Division of Public Health Practice, Harvard School of Public Health, 677 Huntington Ave., Boston, MA 02115, USA. esavoia@hsph.harvard.edu
Testa
Marcia A
MA
Biddinger
Paul D
PD
Cadigan
Rebecca O
RO
Koh
Howard
H
Campbell
Paul
P
Stoto
Michael A
MA
D004189:Disaster Planning / Q000592:standards*; D005069:Evaluation Studies as Topic; D006801:Humans; D007318:Inservice Training; D007857:Leadership; D008281:Maine; D008404:Massachusetts; D011635:Public Health Administration / Q000592:standards*; D011795:Surveys and Questionnaires / Q000592:standards*; D013647:Task Performance and Analysis