title: External validation of Global Evaluative Assessment of Robotic Skills (GEARS).
abstract: BACKGROUND
  We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model.
  
  
  METHODS
  Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal-Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman's rank correlation measured interobserver reliability; and Cronbach's alpha was used to assess internal consistency.
  
  
  RESULTS
  Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93).
  
  
  CONCLUSIONS
  In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized assessment tool, GEARS maintained consistency and reliability for an in vivo robotic surgical task and may be applied for skills evaluation in a broad range of robotic procedures.
authors.affiliation: Department of Urology, Methodist Institute for Technology, Innovation, and Education, Houston Methodist Hospital, 6560 Fannin Street, Suite 2100, Houston, TX, 77030, USA.|Scott Department of Urology, Baylor College of Medicine, Houston, TX, USA.
authors.forename: Monty A
authors.initials: MA
authors.lastname: Aghazadeh
authors.affiliation: USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
authors.forename: Isuru S
authors.initials: IS
authors.lastname: Jayaratna
authors.affiliation: USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Hung
authors.affiliation: Scott Department of Urology, Baylor College of Medicine, Houston, TX, USA.
authors.forename: Michael M
authors.initials: MM
authors.lastname: Pan
authors.affiliation: USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
authors.forename: Mihir M
authors.initials: MM
authors.lastname: Desai
authors.affiliation: USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
authors.forename: Inderbir S
authors.initials: IS
authors.lastname: Gill
authors.affiliation: Department of Urology, Methodist Institute for Technology, Innovation, and Education, Houston Methodist Hospital, 6560 Fannin Street, Suite 2100, Houston, TX, 77030, USA. acg622@gmail.com.
authors.forename: Alvin C
authors.initials: AC
authors.lastname: Goh
mesh_terms: D000328:Adult; D000818:Animals; D002983:Clinical Competence; D003430:Cross-Sectional Studies; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D015203:Reproducibility of Results; D065287:Robotic Surgical Procedures / Q000193:education*; D066231:Surgeons / Q000193:education; D013552:Swine; D013647:Task Performance and Analysis; D014481:United States
keywords: Clinical competence; Education; Robotics; Validation studies