title: Setting Performance Standards for Technical and Nontechnical Competence in General Surgery.
abstract: The objectives of this study were to (1) create a technical and nontechnical performance standard for the laparoscopic cholecystectomy, (2) assess the classification accuracy and (3) credibility of these standards, (4) determine a trainees' ability to meet both standards concurrently, and (5) delineate factors that predict standard acquisition.
  
  
  
  Scores on performance assessments are difficult to interpret in the absence of established standards.
  
  
  
  Trained raters observed General Surgery residents performing laparoscopic cholecystectomies using the Objective Structured Assessment of Technical Skill (OSATS) and the Objective Structured Assessment of Non-Technical Skills (OSANTS) instruments, while as also providing a global competent/noncompetent decision for each performance. The global decision was used to divide the trainees into 2 contrasting groups and the OSATS or OSANTS scores were graphed per group to determine the performance standard. Parametric statistics were used to determine classification accuracy and concurrent standard acquisition, receiver operator characteristic (ROC) curves were used to delineate predictive factors.
  
  
  
  Thirty-six trainees were observed 101 times. The technical standard was an OSATS of 21.04/35.00 and the nontechnical standard an OSANTS of 22.49/35.00. Applying these standards, competent/noncompetent trainees could be discriminated in 94% of technical and 95% of nontechnical performances (P < 0.001). A 21% discordance between technically and nontechnically competent trainees was identified (P < 0.001). ROC analysis demonstrated case experience and trainee level were both able to predict achieving the standards with an area under the curve (AUC) between 0.83 and 0.96 (P < 0.001).
  
  
  
  The present study presents defensible standards for technical and nontechnical performance. Such standards are imperative to implementing summative assessments into surgical training.
authors.affiliation: *Department of Surgery, University of Toronto, Ontario, Canada †Faculty of Health Science and Medicine, Bond University, Queensland, Australia ‡Department of Surgery, Western University, Ontario, Canada §Department of Surgery, University of Calgary, Alberta, Canada ¶Royal College of Physicians and Surgeons of Canada, Ontario, Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Szasz
authors.forename: Esther M
authors.initials: EM
authors.lastname: Bonrath
authors.forename: Marisa
authors.initials: M
authors.lastname: Louridas
authors.forename: Andras B
authors.initials: AB
authors.lastname: Fecso
authors.forename: Brett
authors.initials: B
authors.lastname: Howe
authors.forename: Adam
authors.initials: A
authors.lastname: Fehr
authors.forename: Michael
authors.initials: M
authors.lastname: Ott
authors.forename: Lloyd A
authors.initials: LA
authors.lastname: Mack
authors.forename: Kenneth A
authors.initials: KA
authors.lastname: Harris
authors.forename: Teodor P
authors.initials: TP
authors.lastname: Grantcharov
mesh_terms: D000328:Adult; D019540:Area Under Curve; D002170:Canada; D017081:Cholecystectomy, Laparoscopic / Q000193:education* / Q000592:standards*; D002983:Clinical Competence; D005260:Female; D006801:Humans; D007396:Internship and Residency; D008297:Male; D012372:ROC Curve; D015203:Reproducibility of Results