title: Classification approach for automatic laparoscopic video database organization.
abstract: OBJECTIVE
  One of the advantages of minimally invasive surgery (MIS) is that the underlying digitization provides invaluable information regarding the execution of procedures in various patient-specific conditions. However, such information can only be obtained conveniently if the laparoscopic video database comes with semantic annotations, which are typically provided manually by experts. Considering the growing popularity of MIS, manual annotation becomes a laborious and costly task. In this paper, we tackle the problem of laparoscopic video classification, which consists of automatically identifying the type of abdominal surgery performed in a video. In addition to performing classifications on the full recordings of the procedures, we also carry out sub-video and video clip classifications. These classifications are carried out to investigate how many frames from a video are needed to get a good classification performance and which parts of the procedures contain more discriminative features.
  
  
  METHODS
  Our classification pipeline is as follows. First, we reject the irrelevant frames from the videos using the color properties of the video frames. Second, we extract visual features from the relevant frames. Third, we quantize the features using several feature encoding methods, i.e., vector quantization, sparse coding (SC), and Fisher encoding. Fourth, we carry out the classification using support vector machines. While the sub-video classification is carried out by uniformly downsampling the video frames, the video clip classification is carried out by taking three parts of the videos (i.e., beginning, middle, and end) and running the classification pipeline separately for every video part. Ultimately, we build our final classification model by combining the features using a multiple kernel learning (MKL) approach.
  
  
  RESULTS
  To carry out the experiments, we use a dataset containing 208 videos of eight different surgeries performed by 10 different surgeons. The results show that SC with K-singular value decomposition (K-SVD) yields the best classification accuracy. The results also demonstrate that the classification accuracy only decreases by 3 % when solely 60 % of the video frames are utilized. Furthermore, it is also shown that the end part of the procedures is the most discriminative part of the surgery. Specifically, by using only the last 20 % of the video frames, a classification accuracy greater than 70 % can be achieved. Finally, the combination of all features yields the best performance of 90.38 % accuracy.
  
  
  CONCLUSIONS
  The SC with K-SVD provides the best representation of our videos, yielding the best accuracies for all features. In terms of information, the end part of the laparoscopic videos is the most discriminative compared to the other parts of the videos. In addition to their good performance individually, the features yield even better classification results when all of them are combined using the MKL approach.
authors.affiliation: ICube Laboratory, University of Strasbourg, CNRS, IHU, Strasbourg, France, twinanda@unistra.fr.
authors.forename: Andru Putra
authors.initials: AP
authors.lastname: Twinanda
authors.forename: Jacques
authors.initials: J
authors.lastname: Marescaux
authors.forename: Michel
authors.initials: M
authors.lastname: de Mathelin
authors.forename: Nicolas
authors.initials: N
authors.lastname: Padoy
mesh_terms: D000005:Abdomen / Q000601:surgery; D000465:Algorithms; D001331:Automation; D003116:Color; D003198:Computer Simulation; D016208:Databases, Factual; D006801:Humans; D010535:Laparoscopy / Q000379:methods*; D019060:Minimally Invasive Surgical Procedures / Q000379:methods*; D015233:Models, Statistical; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D060388:Support Vector Machine; D014741:Video Recording
references.citation: IEEE Trans Med Imaging. 2012 Mar;31(3):637-53
references.pmid: 22057050
references.citation: IEEE Trans Biomed Eng. 2012 Apr;59(4):966-76
references.pmid: 22203700
references.citation: Med Image Comput Comput Assist Interv. 2012;15(Pt 2):592-600
references.pmid: 23286097
references.citation: Med Image Comput Comput Assist Interv. 2010;13(Pt 3):400-7
references.pmid: 20879425
references.citation: Med Image Anal. 2013 Oct;17(7):732-45
references.pmid: 23706754
references.citation: IEEE Trans Pattern Anal Mach Intell. 2014 Mar;36(3):507-20
references.pmid: 24457507
references.citation: Med Image Comput Comput Assist Interv. 2014;17(Pt 3):409-16
references.pmid: 25320826