title: "GIOSAT": a tool to assess CanMEDS competencies during simulated crises.
abstract: OBJECTIVE
  Our objective was to develop and evaluate a Generic Integrated Objective Structured Assessment Tool (GIOSAT) to integrate Medical Expert and intrinsic (non-medical expert) CanMEDS competencies with non-technical skills for crisis simulation.
  
  
  METHODS
  An assessment tool was designed and piloted using two pediatric anesthesia scenarios (laryngospasm and hyperkalemia). Following revision of the tool, we used previously recorded videos of anesthesia residents (n = 50) who managed one of two intraoperative advanced cardiac life support (ACLS) scenarios (ventricular tachycardia or ventricular fibrillation). Four independent trained raters, blinded to the residents' level of training, analyzed the video recordings using the GIOSAT scale. Inter-rater reliability was calculated using intraclass correlations (ICCs) for single raters (single measure) and the average of the four raters (average measure), and construct validity was investigated by correlating GIOSAT scores with postgraduate year of residency (PGY).
  
  
  RESULTS
  Total GIOSAT scores for the ACLS scenarios had single measure ICCs of 0.62 and average measure ICCs of 0.85. Inter-rater reliability was substantial for both Medical Expert and intrinsic competencies (single measure ICCs 0.69 and 0.62, respectively; average measure ICCs 0.90 and 0.82, respectively). We found significant correlations between PGY level and total GIOSAT score (r = 0.36; P = 0.011) and between PGY level and Medical Expert competencies (r = 0.42; P = 0.003); however, correlations were not found between PGY level and intrinsic CanMEDS competencies (r = 0.24; P = 0.09).
  
  
  CONCLUSIONS
  Inter-rater reliability of the total GIOSAT scores using four trained raters was substantial. Significant correlation between PGY and (i) total GIOSAT score and (ii) Medical Expert competencies supports construct validity. Evidence of validity was not obtained for intrinsic CanMEDS competencies.
authors: Department of Anesthesia, Children's Hospital of Eastern Ontario, University of Ottawa, 401 Smyth Road, Ottawa, ON, Canada. vneira@cheo.on.ca
authors: Victor M
authors: VM
authors: Neira
authors: M Dylan
authors: MD
authors: Bould
authors: Amy
authors: A
authors: Nakajima
authors: Sylvain
authors: S
authors: Boet
authors: Nicholas
authors: N
authors: Barrowman
authors: Philipp
authors: P
authors: Mossdorf
authors: Devin
authors: D
authors: Sydor
authors: Amy
authors: A
authors: Roeske
authors: Stephen
authors: S
authors: Noseworthy
authors: Viren
authors: V
authors: Naik
authors: Dermot
authors: D
authors: Doherty
authors: Hilary
authors: H
authors: Writer
authors: Stanley J
authors: SJ
authors: Hamstra
mesh_terms: D000758:Anesthesia / Q000009:adverse effects / Q000379:methods; D000776:Anesthesiology / Q000193:education*; D002170:Canada; D002648:Child; D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods; D005260:Female; D006801:Humans; D006947:Hyperkalemia / Q000628:therapy; D007396:Internship and Residency / Q000592:standards*; D007826:Laryngismus / Q000628:therapy; D015588:Observer Variation; D010865:Pilot Projects; D017180:Tachycardia, Ventricular / Q000628:therapy; D014693:Ventricular Fibrillation / Q000628:therapy; D014741:Video Recording