title: The Simulation Team Assessment Tool (STAT): development, reliability and validation.
abstract: BACKGROUND
  Simulation sessions prepare medical professionals for pediatric emergencies. No validated tools exist to evaluate overall team performance. Our objective was to develop and evaluate the inter-rater reliability and validity of a team performance assessment tool during simulated pediatric resuscitations.
  
  
  METHODS
  We developed the Simulation Team Assessment Tool (STAT) which evaluated 4 domains: basic assessment skills, airway/breathing, circulation, and human factors. Scoring of each element was behaviorally anchored from 0 to 2 points. Two teams of resuscitation experts and two teams of pediatric residents performed the same simulated pediatric resuscitation. Each team was scored by six raters using the STAT. Intraclass correlation coefficients (ICC) were calculated to assess inter-rater reliability. Overall performance and domain scores between expert and resident teams were compared using repeated measures of analysis of variance to assess construct validity.
  
  
  RESULTS
  ICCs for overall performance were 0.81. Domain ICCs were: basic skills 0.73, airway/breathing skills 0.30, circulation skills 0.76, human factors 0.68. Expert versus resident average scores were: overall performance 84% vs. 66% (p=0.02), basic skills 73% vs. 55% (p<0.01); airway 80% vs. 75% (p=0.25), circulation 90% vs. 69% (p=0.02), human factors 89% vs. 66% (p=0.02).
  
  
  CONCLUSIONS
  The STAT's overall performance, basic skills, circulation, and human factors domains had good to excellent inter-rater reliability, discriminating well between expert and resident teams. Similar performance in the airway/breathing domain among all teams magnified the impact of a small number of rater disagreements on the ICC. Additional study is needed to better assess the airway/breathing domain.
authors.affiliation: Emergency Services, Seattle Children’s Hospital, Seattle, WA, 98105, United States. Jennifer.reid@seattlechildrens.org
authors.forename: Jennifer
authors.initials: J
authors.lastname: Reid
authors.forename: Kimberly
authors.initials: K
authors.lastname: Stone
authors.forename: Julie
authors.initials: J
authors.lastname: Brown
authors.forename: Derya
authors.initials: D
authors.lastname: Caglar
authors.forename: Ana
authors.initials: A
authors.lastname: Kobayashi
authors.forename: Mithya
authors.initials: M
authors.lastname: Lewis-Newby
authors.forename: Rebecca
authors.initials: R
authors.lastname: Partridge
authors.forename: Kristy
authors.initials: K
authors.lastname: Seidel
authors.forename: Linda
authors.initials: L
authors.lastname: Quan
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D004630:Emergencies; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D010348:Patient Care Team; D010372:Pediatrics / Q000193:education*; D011787:Quality of Health Care; D015203:Reproducibility of Results; D012151:Resuscitation / Q000193:education / Q000592:standards*