title: Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery.
abstract: OBJECTIVE
  Computer-assisted interventions for enhanced minimally invasive surgery (MIS) require tracking of the surgical instruments. Instrument tracking is a challenging problem in both conventional and robotic-assisted MIS, but vision-based approaches are a promising solution with minimal hardware integration requirements. However, vision-based methods suffer from drift, and in the case of occlusions, shadows and fast motion, they can be subject to complete tracking failure.
  
  
  METHODS
  In this paper, we develop a 2D tracker based on a Generalized Hough Transform using SIFT features which can both handle complex environmental changes and recover from tracking failure. We use this to initialize a 3D tracker at each frame which enables us to recover 3D instrument pose over long sequences and even during occlusions.
  
  
  RESULTS
  We quantitatively validate our method in 2D and 3D with ex vivo data collected from a DVRK controller as well as providing qualitative validation on robotic-assisted in vivo data.
  
  
  CONCLUSIONS
  We demonstrate from our extended sequences that our method provides drift-free robust and accurate tracking. Our occlusion-based sequences additionally demonstrate that our method can recover from occlusion-based failure. In both cases, we show an improvement over using 3D tracking alone suggesting that combining 2D and 3D tracking is a promising solution to challenges in surgical instrument tracking.
authors.affiliation: Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK. xiaofei.du.13@ucl.ac.uk.
authors.forename: Xiaofei
authors.initials: X
authors.lastname: Du
authors.affiliation: Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK.
authors.forename: Maximilian
authors.initials: M
authors.lastname: Allan
authors.affiliation: WIREWAX, London, UK.
authors.forename: Alessio
authors.initials: A
authors.lastname: Dore
authors.affiliation: Centre for Medical Image Computing, Department of Medical Physics, University College London, London, UK.
authors.forename: Sebastien
authors.initials: S
authors.lastname: Ourselin
authors.affiliation: Centre for Medical Image Computing, Department of Medical Physics, University College London, London, UK.
authors.forename: David
authors.initials: D
authors.lastname: Hawkes
authors.affiliation: Division of Surgery and Interventional Science, University College London, London, UK.
authors.forename: John D
authors.initials: JD
authors.lastname: Kelly
authors.affiliation: Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK.
authors.forename: Danail
authors.initials: D
authors.lastname: Stoyanov
mesh_terms: D006801:Humans; D019060:Minimally Invasive Surgical Procedures / Q000295:instrumentation / Q000379:methods*; D009038:Motion; D065287:Robotic Surgical Procedures / Q000295:instrumentation / Q000379:methods*; D025321:Surgery, Computer-Assisted / Q000295:instrumentation / Q000379:methods*; D013525:Surgical Instruments
keywords: Instrument tracking and detection; Minimally invasive surgery; Robot-assisted surgery; Surgical vision