Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool.
OBJECTIVE
To evaluate robotic dry laboratory (dry lab) exercises in terms of their face, content, construct and concurrent validities. To evaluate the applicability of the Global Evaluative Assessment of Robotic Skills (GEARS) tool to assess dry lab performance.


METHODS
Participants were prospectively categorized into two groups: robotic novice (no cases as primary surgeon) and robotic expert (≥30 cases). Participants completed three virtual reality (VR) exercises using the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA), as well as corresponding dry lab versions of each exercise (Mimic Technologies, Seattle, WA, USA) on the da Vinci Surgical System. Simulator performance was assessed by metrics measured on the simulator. Dry lab performance was blindly video-evaluated by expert review using the six-metric GEARS tool. Participants completed a post-study questionnaire (to evaluate face and content validity). A Wilcoxon non-parametric test was used to compare performance between groups (construct validity) and Spearman's correlation coefficient was used to assess simulation to dry lab performance (concurrent validity).


RESULTS
The mean number of robotic cases experienced for novices was 0 and for experts the mean (range) was 200 (30-2000) cases. Expert surgeons found the dry lab exercises both 'realistic' (median [range] score 8 [4-10] out of 10) and 'very useful' for training of residents (median [range] score 9 [5-10] out of 10). Overall, expert surgeons completed all dry lab tasks more efficiently (P < 0.001) and effectively (GEARS total score P < 0.001) than novices. In addition, experts outperformed novices in each individual GEARS metric (P < 0.001). Finally, in comparing dry lab with simulator performance, there was a moderate correlation overall (r = 0.54, P < 0.001). Most simulator metrics correlated moderately to strongly with corresponding GEARS metrics (r = 0.54, P < 0.001).


CONCLUSIONS
The robotic dry lab exercises in the present study have face, content, construct and concurrent validity with the corresponding VR tasks. Until now, the assessment of dry lab exercises has been limited to basic metrics (i.e. time to completion and error avoidance). For the first time, we have shown it is feasibile to apply a global assessment tool (GEARS) to dry lab training.
USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
Patrick
P
Ramos
Jeremy
J
Montez
Adrian
A
Tripp
Casey K
CK
Ng
Inderbir S
IS
Gill
Andrew J
AJ
Hung
D002983:Clinical Competence; D003198:Computer Simulation; D004502:Education, Medical, Continuing / Q000379:methods*; D006801:Humans; D017145:Models, Educational; D011446:Prospective Studies; D015203:Reproducibility of Results; D012371:Robotics / Q000193:education*; D011795:Surveys and Questionnaires; D014481:United States; D013520:Urologic Surgical Procedures / Q000193:education*; D014572:Urology / Q000193:education*
dry lab models; educational models; robotics training; validation studies