title: Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room.
abstract: BACKGROUND
  Nontechnical skills are critical for patient safety in the operating room (OR). As a result, regulatory bodies for accreditation and certification have mandated the integration of these competencies into postgraduate education. A generally accepted approach to the in-training assessment of nontechnical skills, however, is lacking. The goal of the present study was to develop an evidence-based and reliable tool for the in-training assessment of residents' nontechnical performance in the OR.
  
  
  METHODS
  The Objective Structured Assessment of Nontechnical Skills tool was designed as a 5-point global rating scale with descriptive anchors for each item, based on existing evidence-based frameworks of nontechnical skills, as well as resident training requirements. The tool was piloted on scripted videos and refined in an iterative process. The final version was used to rate residents' performance in recorded OR crisis simulations and during live observations in the OR.
  
  
  RESULTS
  A total of 37 simulations and 10 live procedures were rated. Interrater agreement was good for total mean scores, both in simulation and in the real OR, with intraclass correlation coefficients >0.90 in all settings for average and single measures. Internal consistency of the scale was high (Cronbach's alpha = 0.80).
  
  
  CONCLUSIONS
  The Objective Structured Assessment of Nontechnical Skills global rating scale was developed as an evidence-based tool for the in-training assessment of residents' nontechnical performance in the OR. Unique descriptive anchors allow for a criterion-referenced assessment of performance. Good reliability was demonstrated in different settings, supporting applications in research and education.
authors.affiliation: Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada. Electronic address: dedyn@smh.ca.
authors.forename: Nicolas J
authors.initials: NJ
authors.lastname: Dedy
authors.affiliation: Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Szasz
authors.affiliation: Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada.
authors.forename: Marisa
authors.initials: M
authors.lastname: Louridas
authors.affiliation: Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada.
authors.forename: Esther M
authors.initials: EM
authors.lastname: Bonrath
authors.affiliation: Department of Obstetrics and Gynaecology, St. Michael's Hospital, University of Toronto, Toronto, Canada.
authors.forename: Heinrich
authors.initials: H
authors.lastname: Husslein
authors.affiliation: Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada.
authors.forename: Teodor P
authors.initials: TP
authors.lastname: Grantcharov
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement; D019317:Evidence-Based Medicine / Q000379:methods; D005260:Female; D006801:Humans; D007318:Inservice Training / Q000458:organization & administration*; D033183:Interdisciplinary Communication; D007396:Internship and Residency; D008297:Male; D015588:Observer Variation; D009864:Ontario; D009873:Operating Rooms / Q000458:organization & administration*; D010348:Patient Care Team / Q000458:organization & administration; D016544:Patient Simulation; D010865:Pilot Projects; D015203:Reproducibility of Results; D013514:Surgical Procedures, Operative / Q000193:education*