title: A robust method to track colonoscopy videos with non-informative images.
abstract: OBJECTIVE
  Continuously, optical and virtual image alignment can significantly supplement the clinical value of colonoscopy. However, the co-alignment process is frequently interrupted by non-informative images. A video tracking framework to continuously track optical colonoscopy images was developed and tested.
  
  
  METHODS
  A video tracking framework with immunity to non-informative images was developed with three essential components: temporal volume flow, region flow, and incremental egomotion estimation. Temporal volume flow selects two similar images interrupted by non-informative images; region flow measures large visual motion between selected images; and incremental egomotion processing estimates significant camera motion by decomposing each large visual motion vector into a sequence of small optical flow vectors. The framework was extensively evaluated via phantom and colonoscopy image sequences. We constructed two colon-like phantoms, a straight phantom and a curved phantom, to measure actual colonoscopy motion.
  
  
  RESULTS
  In the straight phantom, after 48 frames were excluded, the tracking error was [Formula: see text]3 mm of 16 mm traveled. In the curved phantom, the error was [Formula: see text]4 mm of 23.88 mm traveled after 72 frames were excluded. Through evaluations with clinical sequences, the robustness of the tracking framework was demonstrated on 30 colonoscopy image sequences from 22 different patients. Four specific sequences among these were chosen to illustrate the algorithm's decreased sensitivity to (1) fluid immersion, (2) wall contact, (3) surgery-induced colon deformation, and (4) multiple non-informative image sequences.
  
  
  CONCLUSIONS
  A robust tracking framework for real-time colonoscopy was developed that facilitates continuous alignment of optical and virtual images, immune to non-informative images that enter the video stream. The system was validated in phantom testing and achieved success with clinical image sequences.
authors: Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, MD 20892, USA. jianfei.liu@nih.gov
authors: Jianfei
authors: J
authors: Liu
authors: Kalpathi R
authors: KR
authors: Subramanian
authors: Terry S
authors: TS
authors: Yoo
mesh_terms: D000465:Algorithms; D003108:Colonic Diseases / Q000175:diagnosis*; D003113:Colonoscopy / Q000379:methods*; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D019047:Phantoms, Imaging; D015203:Reproducibility of Results; D014741:Video Recording
references: Comput Aided Surg. 2004;9(5):215-26
references: 16192063
references: J Natl Cancer Inst. 2010 Jan 20;102(2):70-1
references: 20042717
references: AJR Am J Roentgenol. 2009 Nov;193(5):1296-304
references: 19843745
references: Comput Vis Image Underst. 2007 Nov;108(1-2):171-187
references: 18978928
references: IEEE Trans Image Process. 1998;7(3):398-410
references: 18276260
references: IEEE Trans Pattern Anal Mach Intell. 2011 May;33(5):978-94
references: 20714019
references: IEEE Trans Biomed Eng. 2009 Sep;56(9):2190-6
references: 19272904
references: AJR Am J Roentgenol. 2009 Nov;193(5):1291-5
references: 19843744
references: Med Image Anal. 2002 Sep;6(3):321-36
references: 12270236
references: Comput Med Imaging Graph. 2008 Apr;32(3):159-73
references: 18096365
references: Med Image Comput Comput Assist Interv. 2006;9(Pt 1):281-8
references: 17354901
references: IEEE Trans Med Imaging. 2006 Nov;25(11):1462-71
references: 17117775
references: IEEE Trans Ultrason Ferroelectr Freq Control. 2010 Jun;57(6):1347-57
references: 20529710
references: Radiology. 2005 Jul;236(1):3-9
references: 15987959
references: IEEE Trans Pattern Anal Mach Intell. 2011 Mar;33(3):500-13
references: 20714020
references: Med Image Anal. 2007 Apr;11(2):110-27
references: 17329146
references: J Opt Soc Am A. 1985 Feb;2(2):354-60
references: 3973767
references: Med Image Comput Comput Assist Interv. 2005;8(Pt 2):543-50
references: 16686002