Use of a machine learning algorithm to classify expertise: analysis of hand motion patterns during a simulated surgical task.
OBJECTIVE
To test the hypothesis that machine learning algorithms increase the predictive power to classify surgical expertise using surgeons' hand motion patterns.


METHODS
In 2012 at the University of North Carolina at Chapel Hill, 14 surgical attendings and 10 first- and second-year surgical residents each performed two bench model venous anastomoses. During the simulated tasks, the participants wore an inertial measurement unit on the dorsum of their dominant (right) hand to capture their hand motion patterns. The pattern from each bench model task performed was preprocessed into a symbolic time series and labeled as expert (attending) or novice (resident). The labeled hand motion patterns were processed and used to train a Support Vector Machine (SVM) classification algorithm. The trained algorithm was then tested for discriminative/predictive power against unlabeled (blinded) hand motion patterns from tasks not used in the training. The Lempel-Ziv (LZ) complexity metric was also measured from each hand motion pattern, with an optimal threshold calculated to separately classify the patterns.


RESULTS
The LZ metric classified unlabeled (blinded) hand motion patterns into expert and novice groups with an accuracy of 70% (sensitivity 64%, specificity 80%). The SVM algorithm had an accuracy of 83% (sensitivity 86%, specificity 80%).


CONCLUSIONS
The results confirmed the hypothesis. The SVM algorithm increased the predictive power to classify blinded surgical hand motion patterns into expert versus novice groups. With further development, the system used in this study could become a viable tool for low-cost, objective assessment of procedural proficiency in a competency-based curriculum.
Dr. Watson is assistant professor, Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.
Robert A
RA
Watson
D000465:Algorithms; D000714:Anastomosis, Surgical; D001185:Artificial Intelligence; D002983:Clinical Competence; D006225:Hand / Q000502:physiology*; D006801:Humans; D011237:Predictive Value of Tests; D012372:ROC Curve; D012680:Sensitivity and Specificity; D013647:Task Performance and Analysis; D014680:Veins / Q000601:surgery*