title: Task-Level vs. Segment-Level Quantitative Metrics for Surgical Skill Assessment.
abstract: OBJECTIVE
  Task-level metrics of time and motion efficiency are valid measures of surgical technical skill. Metrics may be computed for segments (maneuvers and gestures) within a task after hierarchical task decomposition. Our objective was to compare task-level and segment (maneuver and gesture)-level metrics for surgical technical skill assessment.
  
  
  METHODS
  Our analyses include predictive modeling using data from a prospective cohort study. We used a hierarchical semantic vocabulary to segment a simple surgical task of passing a needle across an incision and tying a surgeon's knot into maneuvers and gestures. We computed time, path length, and movements for the task, maneuvers, and gestures using tool motion data. We fit logistic regression models to predict experience-based skill using the quantitative metrics. We compared the area under a receiver operating characteristic curve (AUC) for task-level, maneuver-level, and gesture-level models.
  
  
  METHODS
  Robotic surgical skills training laboratory.
  
  
  METHODS
  In total, 4 faculty surgeons with experience in robotic surgery and 14 trainee surgeons with no or minimal experience in robotic surgery.
  
  
  RESULTS
  Experts performed the task in shorter time (49.74s; 95% CI = 43.27-56.21 vs. 81.97; 95% CI = 69.71-94.22), with shorter path length (1.63m; 95% CI = 1.49-1.76 vs. 2.23; 95% CI = 1.91-2.56), and with fewer movements (429.25; 95% CI = 383.80-474.70 vs. 728.69; 95% CI = 631.84-825.54) than novices. Experts differed from novices on metrics for individual maneuvers and gestures. The AUCs were 0.79; 95% CI = 0.62-0.97 for task-level models, 0.78; 95% CI = 0.6-0.96 for maneuver-level models, and 0.7; 95% CI = 0.44-0.97 for gesture-level models. There was no statistically significant difference in AUC between task-level and maneuver-level (p = 0.7) or gesture-level models (p = 0.17).
  
  
  CONCLUSIONS
  Maneuver-level and gesture-level metrics are discriminative of surgical skill and can be used to provide targeted feedback to surgical trainees.
authors.affiliation: Department of Computer Science, Johns Hopkins University, Baltimore, Maryland. Electronic address: vedula@jhu.edu.
authors.forename: S Swaroop
authors.initials: SS
authors.lastname: Vedula
authors.affiliation: Department of Computer Science, Johns Hopkins University, Baltimore, Maryland.
authors.forename: Anand
authors.initials: A
authors.lastname: Malpani
authors.affiliation: Department of Computer Science, Johns Hopkins University, Baltimore, Maryland.
authors.forename: Narges
authors.initials: N
authors.lastname: Ahmidi
authors.affiliation: Department of Electrical & Computer Engineering, Johns Hopkins University, Baltimore, Maryland.
authors.forename: Sanjeev
authors.initials: S
authors.lastname: Khudanpur
authors.affiliation: Department of Computer Science, Johns Hopkins University, Baltimore, Maryland.
authors.forename: Gregory
authors.initials: G
authors.lastname: Hager
authors.affiliation: Department of Gynecology & Obstetrics, Johns Hopkins University School of Medicine, Baltimore, Maryland.
authors.forename: Chi Chiung Grace
authors.initials: CC
authors.lastname: Chen
mesh_terms: D000328:Adult; D002983:Clinical Competence; D005260:Female; D006801:Humans; D008297:Male; D011446:Prospective Studies; D065287:Robotic Surgical Procedures / Q000193:education* / Q000592:standards*; D013536:Suture Techniques / Q000193:education*; D013996:Time and Motion Studies
keywords: Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; objective skill assessment; robotic surgical skills; segment-level skill metrics; task decomposition; task-level skill metrics