title: Outcome measures for surgical simulators: is the focus on technical skills the best approach?
abstract: BACKGROUND
  Mastery of operative performance is based on technical skill and intra-operative judgment. However, previous simulation studies have largely focused on technical skills and measures. This study investigates changes in operative performance when assessment and feedback focus on decision making.
  
  
  METHODS
  Using a nonequivalent, pretest/post-test experimental design, 8 senior residents (PGY4-5) performed a laparoscopic ventral hernia repair using a newly developed box-trainer style simulator fabricated to induce surgical decision making. The pretest simulator had a 10 x 10-cm defect 5 cm above the umbilicus. The post-test simulator had a 10 x 10-cm defect in the right upper quadrant. After the pretest, faculty provided immediate feedback on operative decisions that lead to errors. In addition, residents were allowed to visually inspect their repair by removing the box trainer skins. Video-analysis using a 9-item decision making checklist was used to categorize pretest and post-test error differences.
  
  
  RESULTS
  Common errors made during the pretest included improper visualization of the suture passer and improper mesh preparation on the back table. These errors resulted in incomplete hernia repairs by 75% of residents on the pretest. In contrast, 100% of residents successfully completed the more difficult, nonequivalent post-test hernia. Checklist analysis showed residents committed more errors on the pretest resulting in lower performance scores (score = 48.12; SD = 19.26) compared with post-test performance (score = 75.00; SD = 14.39; P < .05).
  
  
  CONCLUSIONS
  Residents' decision-making skills seem to significantly affect operative performance. To facilitate mastery of operative performance, additional research is needed on simulation-based, operative skills measures that focus on intra-operative decision making.
authors.affiliation: Department of Surgery, Northwestern University, 676 North St. Clair, Suite 650, Chicago, IL 60610, USA. drpugh@northwestern.edu
authors.forename: Carla
authors.initials: C
authors.lastname: Pugh
authors.forename: Stephen
authors.initials: S
authors.lastname: Plachta
authors.forename: Edward
authors.initials: E
authors.lastname: Auyang
authors.forename: Aurora
authors.initials: A
authors.lastname: Pryor
authors.forename: Eric
authors.initials: E
authors.lastname: Hungness
mesh_terms: D057189:Checklist; D003162:Competency-Based Education; D003194:Computer-Assisted Instruction; D003657:Decision Making; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000379:methods*; D030141:Feedback, Psychological; D013502:General Surgery / Q000193:education*; D006555:Hernia, Ventral / Q000601:surgery; D006801:Humans; D007396:Internship and Residency; D010535:Laparoscopy