Outcome measures for surgical simulators: is the focus on technical skills the best approach?
BACKGROUND
Mastery of operative performance is based on technical skill and intra-operative judgment. However, previous simulation studies have largely focused on technical skills and measures. This study investigates changes in operative performance when assessment and feedback focus on decision making.


METHODS
Using a nonequivalent, pretest/post-test experimental design, 8 senior residents (PGY4-5) performed a laparoscopic ventral hernia repair using a newly developed box-trainer style simulator fabricated to induce surgical decision making. The pretest simulator had a 10 x 10-cm defect 5 cm above the umbilicus. The post-test simulator had a 10 x 10-cm defect in the right upper quadrant. After the pretest, faculty provided immediate feedback on operative decisions that lead to errors. In addition, residents were allowed to visually inspect their repair by removing the box trainer skins. Video-analysis using a 9-item decision making checklist was used to categorize pretest and post-test error differences.


RESULTS
Common errors made during the pretest included improper visualization of the suture passer and improper mesh preparation on the back table. These errors resulted in incomplete hernia repairs by 75% of residents on the pretest. In contrast, 100% of residents successfully completed the more difficult, nonequivalent post-test hernia. Checklist analysis showed residents committed more errors on the pretest resulting in lower performance scores (score = 48.12; SD = 19.26) compared with post-test performance (score = 75.00; SD = 14.39; P < .05).


CONCLUSIONS
Residents' decision-making skills seem to significantly affect operative performance. To facilitate mastery of operative performance, additional research is needed on simulation-based, operative skills measures that focus on intra-operative decision making.
Pugh
Carla
C
Department of Surgery, Northwestern University, 676 North St. Clair, Suite 650, Chicago, IL 60610, USA. drpugh@northwestern.edu
Plachta
Stephen
S
Auyang
Edward
E
Pryor
Aurora
A
Hungness
Eric
E
D057189:Checklist; D003162:Competency-Based Education; D003194:Computer-Assisted Instruction; D003657:Decision Making; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000379:methods*; D030141:Feedback, Psychological; D013502:General Surgery / Q000193:education*; D006555:Hernia, Ventral / Q000601:surgery; D006801:Humans; D007396:Internship and Residency; D010535:Laparoscopy