title: Prospective observational study to evaluate NOTSS (Non-Technical Skills for Surgeons) for assessing trainees' non-technical performance in the operating theatre.
abstract: BACKGROUND
  Most surgical assessment has been aimed at technical proficiency. However, non-technical skills also affect patient safety and clinical effectiveness. The NOTSS (Non-Technical Skills for Surgeons) assessment instrument was developed specifically to assess the non-technical skills of individual surgeons in the operating theatre. This study evaluated NOTSS as a real-world assessment, with a mix of minimally trained assessors. The evaluation criteria were feasibility, validity and psychometric reliability.
  
  
  METHODS
  In a standard evaluation of NOTSS, 56 anaesthetists, 39 scrub nurses, two surgical care practitioners and three independent assessors provided 715 assessments of 404 surgical cases of 15 index procedures across six specialties performed by 85 surgical trainees.
  
  
  RESULTS
  The assessment was feasible, but important implementation challenges were highlighted. Most respondents considered the method valid, but with reservations about assessing cognition. The factor structure of scores, and their positive relationships with other measures of experience and performance, supported validity. Trainees' non-technical skill scores were relatively procedure-independent and achieved good reliability (generalizability coefficient 0·8 or more) when six to eight assessors observed one case each.
  
  
  CONCLUSIONS
  Minimally trained assessors, who are typically present in operating theatres, were sufficiently discriminating and consistent in their judgements of trainee surgeons' non-technical skills to provide reliable scores based on an achievable number of observations.
authors.affiliation: Academic Unit of Medical Education, University of Sheffield, Sheffield, UK.
authors.forename: J
authors.initials: J
authors.lastname: Crossley
authors.forename: J
authors.initials: J
authors.lastname: Marriott
authors.forename: H
authors.initials: H
authors.lastname: Purdie
authors.forename: J D
authors.initials: JD
authors.lastname: Beard
mesh_terms: D000328:Adult; D000776:Anesthesiology / Q000592:standards*; D001291:Attitude of Health Personnel; D002983:Clinical Competence / Q000592:standards*; D005240:Feasibility Studies; D005260:Female; D013502:General Surgery / Q000592:standards*; D006801:Humans; D008297:Male; D008875:Middle Aged; D013527:Perioperative Nursing / Q000592:standards*; D011364:Professional Practice / Q000592:standards; D011446:Prospective Studies; D015203:Reproducibility of Results