Evaluating minimally invasive surgery training using low-cost mechanical simulations.
BACKGROUND
The goal of this study was to develop, test, and validate the efficacy of inexpensive mechanical minimally invasive surgery (MIS) model simulations for training faculty, residents, and medical students. We sought to demonstrate that trained and experienced MIS surgeon raters could reliably rate the MIS skills acquired during these simulations.


METHODS
We developed three renewable models that represent difficult or challenging segments of laparoscopic procedures; laparoscopic appendectomy (LA), laparoscopic cholecystectomy (LC), and laparoscopic inguinal hernia (LH). We videotaped 10 students, 12 surgical residents, and 1 surgeon receiving training on each of the models and again during their posttraining evaluation session. Five MIS surgeons then assessed the evaluation session performance. For each simulation, we asked them to rate overall competence (COM) and four skills: clinical judgment (respect for tissue) (CJ), dexterity (economy of movement) (DEX), serial/simultaneous complexity (SSC), and spatial orientation (SO). We computed intraclass correlation (ICC) coefficients to determine the extent of agreement (i.e., reliability) among ratings.


RESULTS
We obtained ICC values of 0.74, 0.84, and 0.81 for COM ratings on LH, LC, and LA, respectively. We also obtained the following ICC values for the same three models: CJ, 0.75, 0.83, and 0.89; DEX, 0.88, 0.86, and 0.89; SSC, 0.82, 0.82, and 0.82; and SO, 0.86, 0.86, and 0.87, respectively.


CONCLUSIONS
We obtained very high reliability of performance ratings for competence and surgical skills using a mechanical simulator. Typically, faculty evaluations of residents in the operating room are much less reliable. In contrast, when faculty members observe residents in a controlled, standardized environment, their ratings can be very reliable.
Adrales
G L
GL
Center for Minimally Invasive Surgery, University of Kentucky, Lexington, KY 40536-0298, USA.
Chu
U B
UB
Witzke
D B
DB
Donnelly
M B
MB
Hoskins
D
D
Mastrangelo
M J
MJ
Gandsas
A
A
Park
A E
AE
D002983:Clinical Competence; D018961:Educational Technology; D006801:Humans; D010535:Laparoscopy; D019060:Minimally Invasive Surgical Procedures / Q000193:education*; D017145:Models, Educational; D015203:Reproducibility of Results; D013664:Teaching Materials