Structured learning for robotic surgery utilizing a proficiency score: a pilot study.
OBJECTIVE
We evaluated feasibility and benefit of implementing structured learning in a robotics program. Furthermore, we assessed validity of a proficiency assessment tool for stepwise graduation.


METHODS
Teaching cases included robotic radical prostatectomy and partial nephrectomy. Procedure steps were categorized: basic, intermediate, and advanced. An assessment tool ["proficiency score" (PS)] was developed to evaluate ability to safely and autonomously complete a step. Graduation required a passing PS (PS ≥ 3) on three consecutive attempts. PS and validated global evaluative assessment of robotic skills (GEARS) were evaluated for completed steps. Linear regression was utilized to determine postgraduate year/PS relationship (construct validity). Spearman's rank correlation coefficient measured correlation between PS and GEARS evaluations (concurrent validity). Intraclass correlation (ICC) evaluated PS agreement between evaluator classes.


RESULTS
Twenty-one robotic trainees participated within the pilot program, completing a median of 14 (2-69) cases each. Twenty-three study evaluators scored 14 (1-60) cases. Over 4 months, 229/294 (78 %) cases were designated "teaching" cases. Residents completed 91 % of possible evaluations; faculty completed 78 %. Verbal and quantitative feedback received by trainees increased significantly (p = 0.002, p < 0.001, respectively). Average PS increased with PGY (post-graduate year) for basic and intermediate steps (regression slopes: 0.402 (p < 0.0001), 0.323 (p < 0.0001), respectively) (construct validation). Overall, PS correlated highly with GEARS (ρ = 0.81, p < 0.0001) (concurrent validity). ICC was 0.77 (95 % CI 0.61-0.88) for resident evaluations.


CONCLUSIONS
Structured learning can be implemented in an academic robotic program with high levels of trainee and evaluator participation, encouraging both quantitative and verbal feedback. A proficiency assessment tool developed for step-specific proficiency has construct and concurrent validity.
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA. Andrew.Hung@med.usc.edu.
Andrew J
AJ
Hung
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Thomas
T
Bottyan
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Thomas G
TG
Clifford
Department of Psychology, University of Southern California, Los Angeles, CA, USA.
Sarfaraz
S
Serang
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Zein K
ZK
Nakhoda
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Swar H
SH
Shah
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Hana
H
Yokoi
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Monish
M
Aron
Center for Robotic Simulation and Education, USC Institute of Urology, Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, 1441 Eastlake Avenue Suite 7416, Los Angeles, CA, 90089, USA.
Inderbir S
IS
Gill
D002983:Clinical Competence; D003162:Competency-Based Education; D003479:Curriculum; D005180:Faculty, Medical; D005240:Feasibility Studies; D006801:Humans; D007396:Internship and Residency; D016014:Linear Models; D009392:Nephrectomy / Q000193:education*; D010865:Pilot Projects; D011468:Prostatectomy / Q000193:education*; D065287:Robotic Surgical Procedures / Q000193:education*; D014572:Urology / Q000193:education*
Proficiency; Robotic curriculum; Robotic training