title: Surgical gesture segmentation and recognition.
abstract: Automatic surgical gesture segmentation and recognition can provide useful feedback for surgical training in robotic surgery. Most prior work in this field relies on the robot's kinematic data. Although recent work [1,2] shows that the robot's video data can be equally effective for surgical gesture recognition, the segmentation of the video into gestures is assumed to be known. In this paper, we propose a framework for joint segmentation and recognition of surgical gestures from kinematic and video data. Unlike prior work that relies on either frame-level kinematic cues, or segment-level kinematic or video cues, our approach exploits both cues by using a combined Markov/semi-Markov conditional random field (MsM-CRF) model. Our experiments show that the proposed model improves over a Markov or semi-Markov CRF when using video data alone, gives results that are comparable to state-of-the-art methods on kinematic data alone, and improves over state-of-the-art methods when combining kinematic and video data.
authors: Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA.
authors: Lingling
authors: L
authors: Tao
authors: Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA.
authors: Luca
authors: L
authors: Zappella
authors: Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA.
authors: Gregory D
authors: GD
authors: Hager
authors: Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA.
authors: René
authors: R
authors: Vidal
mesh_terms: D001132:Arm / Q000033:anatomy & histology / Q000503:physiopathology*; D003198:Computer Simulation; D005868:Gestures; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000379:methods*; D008954:Models, Biological; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012371:Robotics / Q000379:methods*; D012680:Sensitivity and Specificity; D025321:Surgery, Computer-Assisted / Q000379:methods*