title: Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.
abstract: BACKGROUND
  Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention.
  
  
  METHODS
  We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects.
  
  
  RESULTS
  Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98.
  
  
  CONCLUSIONS
  A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.
authors.affiliation: From the USAF Center for Sustainment of Trauma and Readiness Skills (S.S., J.P.); Department of Anatomy and Neurobiology (E.G.), Shock Trauma Anesthesiology Research Center (K.P., C.F.M.), Department of Epidemiology (H.C.), and Department of Surgery (S.H.), University of Maryland School of Medicine, Baltimore; and The Norman M. Rich Department of Surgery (M.B.), Uniformed Services University of Health Sciences, Bethesda, Maryland; Department of Psychology (V.S.), Wright State University, Dayton, Ohio; Department of Surgery (B.S.), George Washington University, Washington, District of Columbia.
authors.forename: Stacy
authors.initials: S
authors.lastname: Shackelford
authors.forename: Evan
authors.initials: E
authors.lastname: Garofalo
authors.forename: Valerie
authors.initials: V
authors.lastname: Shalin
authors.forename: Kristy
authors.initials: K
authors.lastname: Pugh
authors.forename: Hegang
authors.initials: H
authors.lastname: Chen
authors.forename: Jason
authors.initials: J
authors.lastname: Pasley
authors.forename: Babak
authors.initials: B
authors.lastname: Sarani
authors.forename: Sharon
authors.initials: S
authors.lastname: Henry
authors.forename: Mark
authors.initials: M
authors.lastname: Bowyer
authors.forename: Colin F
authors.initials: CF
authors.lastname: Mackenzie
mesh_terms: D000328:Adult; D057189:Checklist; D002983:Clinical Competence / Q000592:standards; D004521:Educational Measurement / Q000379:methods; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D013647:Task Performance and Analysis; D014194:Traumatology / Q000193:education*