title: An Electronic Competency-Based Evaluation Tool for Assessing Humanitarian Competencies in a Simulated Exercise.
abstract: METHODS
  The evaluation tool was first derived from the formerly Consortium of British Humanitarian Agencies' (CBHA; United Kingdom), now "Start Network's," Core Humanitarian Competency Framework and formatted in an electronic data capture tool that allowed for offline evaluation. During a 3-day humanitarian simulation event, participants in teams of eight to 10 were evaluated individually at multiple injects by trained evaluators. Participants were assessed on five competencies and a global rating scale. Participants evaluated both themselves and their team members using the same tool at the end of the simulation exercise (SimEx).
  
  
  RESULTS
  All participants (63) were evaluated. A total of 1,008 individual evaluations were completed. There were 90 (9.0%) missing evaluations. All 63 participants also evaluated themselves and each of their teammates using the same tool. Self-evaluation scores were significantly lower than peer-evaluations, which were significantly lower than evaluators' assessments. Participants with a medical degree, and those with humanitarian work experience of one month or more, scored significantly higher on all competencies assessed by evaluators compared to other participants. Participants with prior humanitarian experience scored higher on competencies regarding operating safely and working effectively as a team member.
  
  
  CONCLUSIONS
  This study presents a novel electronic evaluation tool to assess individual performance in five of six globally recognized humanitarian competency domains in a 3-day humanitarian SimEx. The evaluation tool provides a standardized approach to the assessment of humanitarian competencies that cannot be evaluated through knowledge-based testing in a classroom setting. When combined with testing knowledge-based competencies, this presents an approach to a comprehensive competency-based assessment that provides an objective measurement of competency with respect to the competencies listed in the Framework. There is an opportunity to advance the use of this tool in future humanitarian training exercises and potentially in real time, in the field. This could impact the efficiency and effectiveness of humanitarian operations. Evans AB , Hulme JM , Nugus P , Cranmer HH , Coutu M , Johnson K . An electronic competency-based evaluation tool for assessing humanitarian competencies in a simulated exercise. Prehosp Disaster Med. 2017;32(3):253-260.
authors.affiliation: 1Emergency Department,The Hospital for Sick Children,Toronto,Canada.
authors.forename: Andrea B
authors.initials: AB
authors.lastname: Evans
authors.affiliation: 3Emergency Department,University Health Network,University of Toronto,Toronto,Canada.
authors.forename: Jennifer M
authors.initials: JM
authors.lastname: Hulme
authors.affiliation: 5Centre for Medical Education,McGill University,Montreal,Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Nugus
authors.affiliation: 7Global Health,Massachusetts General Hospital,Boston,MassachusettsUSA.
authors.forename: Hilarie H
authors.initials: HH
authors.lastname: Cranmer
authors.affiliation: 8McGill Humanitarian Studies Initiative,McGill University,Montreal,Canada.
authors.forename: Melanie
authors.initials: M
authors.lastname: Coutu
authors.affiliation: 6Department of Family Medicine,McGill University,Montreal,Canada.
authors.forename: Kirsten
authors.initials: K
authors.lastname: Johnson
mesh_terms: D000328:Adult; D000533:Altruism; D019985:Benchmarking; D003198:Computer Simulation; D004190:Disasters; D005260:Female; D006801:Humans; D020407:Internet; D008297:Male; D011792:Quebec; D015203:Reproducibility of Results; D018635:Rescue Work / Q000458:organization & administration / Q000592:standards*; D000078329:Workforce
keywords:  CBHA Consortium of British Humanitarian Agencies;  MD Medical Doctor;  SimEx simulation exercise; competency; evaluation; humanitarian response; humanitarian training; humanitarian workers; simulation exercise