Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents.
BACKGROUND
In response to new Accreditation Council for Graduate Medical Education requirements about simulation skill laboratories, programs are incorporating simulation into residents' training. Despite substantial research on simulators, few data exist to support the effectiveness of simulation skills curricula. We report on an Objective Structured Assessment of Technical Skills (OSATS) used to assess residents' needs and evaluate a curriculum designed to increase proficiency.


METHODS
The five-session (10-week) curriculum covered asepsis, skin preparation, gowning, gloving, knot-tying, suturing, and excision. Performance on a 20-minute OSATS station was measured by unblinded raters using a task-specific checklist and seven global rating scales. Interns' pre-post improvement was assessed using paired t-tests. PGY2 and PGY3 residents were used as nonequivalent controls; their scores set a benchmark for PGY1 residents postcurriculum. Percentage of possible points earned was compared with a 75% "needs" criterion; ANOVA was used to assess group differences at the p < 0.05 level.


RESULTS
Seven PGY2 and 6 PGY3 residents took the OSATS; 24 of 25 PGY1s completed both the baseline and postcurriculum OSATS. At baseline, PGY1 mean percent correct total score was 49%; they performed considerably below PGY2 (68%) and PGY3 (74%) residents. PGY1 scores improved significantly after 10 weeks (p < or = 0.001). When their postcurriculum scores were compared with PGY2 and PGY3 resident benchmarks, there were no significant differences in checklist (p = 0.38), global item (p = 0.29), or total scores (p = 0.45).


CONCLUSIONS
Our results suggest that the simulation curriculum helped PGY1 residents attain basic surgical skills at levels consistent with PGY2 and PGY3 residents as measured by an OSATS. Only PGY3 residents performed at the 75% criterion.
Chipman
Jeffrey G
JG
Department of Surgery, University of Minnesota, 420 Delaware Street SE, Minneapolis, MN 55455, USA. chipm001@umn.edu
Schmitz
Constance C
CC
D000704:Analysis of Variance; D002983:Clinical Competence; D003479:Curriculum; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D016544:Patient Simulation; D015203:Reproducibility of Results; D014481:United States