title: Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity?
abstract: BACKGROUND
  Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation.
  
  
  OBJECTIVE
  This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity.
  
  
  METHODS
  General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LAP Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p < 0.05 was significant).
  
  
  RESULTS
  Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p < 0.05). The most discriminatory procedural metrics (p < 0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045).
  
  
  CONCLUSIONS
  Individual variability within the general surgeon and colorectal surgeon groups was not accounted for.
  
  
  CONCLUSIONS
  The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.
authors.affiliation: 1Division of Colorectal Surgery, University Hospitals Case Medical Center, Cleveland, Ohio 2Department of Digestive Surgery, University Hospitals of Bordeaux, Bordeaux, France 3Division of Colorectal Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California 4Division of Colorectal Surgery, Veterans Administration Gulf Coast Veterans Health Care System, Biloxi, Mississippi.
authors.forename: Skandan
authors.initials: S
authors.lastname: Shanmugan
authors.forename: Fabien
authors.initials: F
authors.lastname: Leblanc
authors.forename: Anthony J
authors.initials: AJ
authors.lastname: Senagore
authors.forename: C Neal
authors.initials: CN
authors.lastname: Ellis
authors.forename: Sharon L
authors.initials: SL
authors.lastname: Stein
authors.forename: Sadaf
authors.initials: S
authors.lastname: Khan
authors.forename: Conor P
authors.initials: CP
authors.lastname: Delaney
authors.forename: Bradley J
authors.initials: BJ
authors.lastname: Champagne
mesh_terms: D002983:Clinical Competence; D003082:Colectomy / Q000193:education*; D003198:Computer Simulation; D004210:Dissection / Q000193:education; D006801:Humans; D010535:Laparoscopy / Q000193:education*; D009048:Motor Skills; D061646:Operative Time; D015203:Reproducibility of Results; D013647:Task Performance and Analysis; D014584:User-Computer Interface