title: Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training.
abstract: OBJECTIVE
  To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool.
  
  
  BACKGROUND
  Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated.
  
  
  METHODS
  Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session.
  
  
  RESULTS
  All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert.
  
  
  CONCLUSIONS
  It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds.
authors.affiliation: Centre for Patient Safety and Service Quality, Department of Surgery and Cancer, Imperial College, London, UK. s.russ@imperial.ac.uk
authors.forename: Stephanie
authors.initials: S
authors.lastname: Russ
authors.forename: Louise
authors.initials: L
authors.lastname: Hull
authors.forename: Shantanu
authors.initials: S
authors.lastname: Rout
authors.forename: Charles
authors.initials: C
authors.lastname: Vincent
authors.forename: Ara
authors.initials: A
authors.lastname: Darzi
authors.forename: Nick
authors.initials: N
authors.lastname: Sevdalis
mesh_terms: D002138:Calibration; D002983:Clinical Competence; D003299:Cooperative Behavior; D005240:Feasibility Studies; D013502:General Surgery / Q000592:standards*; D006801:Humans; D007400:Interprofessional Relations; D059032:Learning Curve; D015588:Observer Variation; D010348:Patient Care Team / Q000592:standards*; D017064:Process Assessment, Health Care / Q000379:methods*; D011446:Prospective Studies; D016037:Single-Blind Method; D013663:Teaching