Shot boundary detection in endoscopic surgery videos using a variational Bayesian framework.
OBJECTIVE
Over the last decade, the demand for content management of video recordings of surgical procedures has greatly increased. Although a few research methods have been published toward this direction, the related literature is still in its infancy. In this paper, we address the problem of shot detection in endoscopic surgery videos, a fundamental step in content-based video analysis.


METHODS
The video is first decomposed into short clips that are processed sequentially. After feature extraction, we employ spatiotemporal Gaussian mixture models (GMM) for each clip and apply a variational Bayesian (VB) algorithm to approximate the posterior distribution of the model parameters. The proper number of components is handled automatically by the VBGMM algorithm. The estimated components are matched along the video sequence via their Kullback-Leibler divergence. Shot borders are defined when component tracking fails, signifying a different visual appearance of the surgical scene.


RESULTS
Experimental evaluation was performed on laparoscopic videos containing a variable number of shots. Performance was measured via precision, recall, coverage and overflow metrics. The proposed method was compared with GMM and a shot detection method based on spatiotemporal motion differences (MotionDiff). The results demonstrate that VBGMM has higher performance than all other methods for most assessment metrics: precision and recall >80 %, coverage: 84 %. Overflow for VBGMM was worse than MotionDiff (37 vs. 27 %).


CONCLUSIONS
The proposed method generated promising results for shot border detection. Spatiotemporal modeling via VBGMMs provides a means to explore additional applications such as component tracking.
Simulation Center, Laboratory of Medical Physics, Medical School, National and Kapodistrian University of Athens, Athens, Greece. cloukas@med.uoa.gr.|Medical Physics Lab-Simulation Center, School of Medicine, University of Athens, Mikras Asias 75 str., 11527, Athens, Greece. cloukas@med.uoa.gr.
Constantinos
C
Loukas
Simulation Center, Laboratory of Medical Physics, Medical School, National and Kapodistrian University of Athens, Athens, Greece.
Nikolaos
N
Nikiteas
1st Department of Surgery, Laiko General Hospital, University of Athens, Athens, Greece.
Dimitrios
D
Schizas
Simulation Center, Laboratory of Medical Physics, Medical School, National and Kapodistrian University of Athens, Athens, Greece.
Evangelos
E
Georgiou
D000465:Algorithms; D001499:Bayes Theorem; D017081:Cholecystectomy, Laparoscopic / Q000379:methods*; D004724:Endoscopy / Q000379:methods*; D006801:Humans; D008962:Models, Theoretical; D016011:Normal Distribution; D014741:Video Recording / Q000379:methods
Border detection; Shot detection; Surgery; Tracking; Variational Bayes; Video content analysis