title: Which skills really matter? proving face, content, and construct validity for a commercial robotic simulator.
abstract: BACKGROUND
  A novel computer simulator is now commercially available for robotic surgery using the da Vinci(®) System (Intuitive Surgical, Sunnyvale, CA). Initial investigations into its utility have been limited due to a lack of understanding of which of the many provided skills modules and metrics are useful for evaluation. In addition, construct validity testing has been done using medical students as a "novice" group-a clinically irrelevant cohort given the complexity of robotic surgery. This study systematically evaluated the simulator's skills tasks and metrics and established face, content, and construct validity using a relevant novice group.
  
  
  METHODS
  Expert surgeons deconstructed the task of performing robotic surgery into eight separate skills. The content of the 33 modules provided by the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA) was then evaluated for these deconstructed skills and 8 of the 33 determined to be unique. These eight tasks were used for evaluating the performance of 46 surgeons and trainees on the simulator (25 novices, 8 intermediate, and 13 experts). Novice surgeons were general surgery and urology residents or practicing surgeons with clinical experience in open and laparoscopic surgery but limited exposure to robotics. Performance was measured using 85 metrics across all eight tasks.
  
  
  RESULTS
  Face and content validity were confirmed using global rating scales. Of the 85 metrics provided by the simulator, 11 were found to be unique, and these were used for further analysis. Experts performed significantly better than novices in all eight tasks and for nearly every metric. Intermediates were inconsistently better than novices, with only four tasks showing a significant difference in performance. Intermediate and expert performance did not differ significantly.
  
  
  CONCLUSIONS
  This study systematically determined the important modules and metrics on the da Vinci Skills Simulator and used them to demonstrate face, content, and construct validity with clinically relevant novice, intermediate, and expert groups. These data will be used to develop proficiency-based training programs on the simulator and to investigate predictive validity.
authors: The Methodist Hospital Department of Surgery, The Methodist Institute for Technology Innovation and Education (MITIE SM), 6550 Fannin Street, Suite 1661A, Houston, TX 77030, USA. clyons@tmhs.org
authors: Calvin
authors: C
authors: Lyons
authors: David
authors: D
authors: Goldfarb
authors: Stephen L
authors: SL
authors: Jones
authors: Niraj
authors: N
authors: Badhiwala
authors: Brian
authors: B
authors: Miles
authors: Richard
authors: R
authors: Link
authors: Brian J
authors: BJ
authors: Dunkin
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D003198:Computer Simulation / Q000592:standards*; D004867:Equipment Design; D013502:General Surgery / Q000193:education / Q000295:instrumentation / Q000592:standards*; D006801:Humans; D011597:Psychomotor Performance / Q000502:physiology; D012371:Robotics / Q000193:education / Q000295:instrumentation*
references: J Endourol. 2009 Mar;23(3):503-8
references: 19265469
references: J Urol. 2012 Mar;187(3):998-1002
references: 22264455
references: J Urol. 2011 Sep;186(3):1019-24
references: 21784469
references: J Endourol. 2007 Mar;21(3):244-7
references: 17444766
references: J Robot Surg. 2009 Oct;3(3):165
references: 27638373
references: Urology. 2009 Jun;73(6):1288-92
references: 19362352
references: J Urol. 2002 Sep;168(3):945-9
references: 12187196
references: J Urol. 2012 Jan;187(1):247-52
references: 22099993
references: Am J Surg. 2010 Apr;199(4):529-41
references: 20080227