title: Standardized assessment for evaluation of team skills: validity and feasibility.
abstract: BACKGROUND
  The authors developed a Standardized Assessment for Evaluation of Team Skills (SAFE-TeamS) in which actors portray health care team members in simulated challenging teamwork scenarios. Participants are scored on scenario-specific ideal behaviors associated with assistance, conflict resolution, communication, assertion, and situation assessment. This research sought to provide evidence of the validity and feasibility of SAFE-TeamS as a tool to support the advancement of science related to team skills training.
  
  
  METHODS
  Thirty-eight medical and nursing students were assessed using SAFE-TeamS before and after team skills training. The SAFE-TeamS pretraining and posttraining scores were compared, and participants were surveyed. Generalizability analysis was used to estimate the variance in scores associated with the following: examinee, scenario, rater, pretraining/posttraining, examinee type, rater type (actor-live vs. external rater-videotape), actor team, and scenario order.
  
  
  RESULTS
  The SAFE-TeamS scores reflected improvement after training and were sensitive to individual differences. Score variance due to rater was low. Variance due to scenario was moderate. Estimates of relative reliability for 2 raters and 8 scenarios ranged from 0.6 to 0.7. With fixed scenarios and raters, 2 raters and 2 scenarios, reliability is greater than 0.8. Raters believed SAFE-TeamS assessed relevant team skills. Examinees' responses were mixed.
  
  
  CONCLUSIONS
  The SAFE-TeamS was sensitive to individual differences and team skill training, providing evidence for validity. It is not clear whether different scenarios measure different skills and whether the scenarios cover the necessary breadth of skills. Use of multiple scenarios will support assessment across a broader range of skills. Future research is required to determine whether assessments using SAFE-TeamS will translate to performance in clinical practice.
authors.affiliation: From the Research Institute (M.C.W.), Saint Alphonsus Health System, Boise, ID; Trinity Institute for Health and Community Benefit (M.C.W.), Trinity Health, Livonia, MI; Department of Anesthesiology (J.M.T., N.S., G.H.,B.P.-B.) and Human Simulation and Patient Safety Center (J.M.T., N.S., G.H.), School of Medicine, Duke University, Durham, NC; NC Center for Hospital Quality and Patient Safety (L.M.), North Carolina Hospital Association, Cary, NC.
authors.forename: Melanie C
authors.initials: MC
authors.lastname: Wright
authors.forename: Noa
authors.initials: N
authors.lastname: Segall
authors.forename: Gene
authors.initials: G
authors.lastname: Hobbs
authors.forename: Barbara
authors.initials: B
authors.lastname: Phillips-Bute
authors.forename: Laura
authors.initials: L
authors.lastname: Maynard
authors.forename: Jeffrey M
authors.initials: JM
authors.lastname: Taekman
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003198:Computer Simulation; D004521:Educational Measurement / Q000379:methods / Q000592:standards; D005240:Feasibility Studies; D006801:Humans; D033183:Interdisciplinary Communication; D010348:Patient Care Team / Q000458:organization & administration / Q000592:standards*; D015397:Program Evaluation / Q000379:methods; D011785:Quality Assurance, Health Care / Q000379:methods*; D015203:Reproducibility of Results; D013337:Students, Medical; D013338:Students, Nursing; D014743:Videotape Recording