title: ToF meets RGB: novel multi-sensor super-resolution for hybrid 3-D endoscopy.
abstract: 3-D endoscopy is an evolving field of research with the intention to improve safety and efficiency of minimally invasive surgeries. Time-of-Flight (ToF) imaging allows to acquire range data in real-time and has been engineered into a 3-D endoscope in combination with an RGB sensor (640x480px) as a hybrid imaging system, recently. However, the ToF sensor suffers from a low spatial resolution (640 x 480 px) and a poor signal-to-noise ratio. In this paper, we propose a novel multi-frame super-resolution framework to improve range images in a ToF/RGB multi-sensor setup. Our approach exploits high-resolution RGB data to estimate subpixel motion used as a cue for range super-resolution. The underlying non-parametric motion model based on optical flow makes the method applicable to endoscopic scenes with arbitrary endoscope movements. The proposed method was evaluated on synthetic and real images. Our approach improves the peak-signal-to-noise ratio by 1.6 dB and structural similarity by 0.02 compared to single-sensor super-resolution.
authors: Pattern Recognition Lab, Friedrich-Alexander-Universitäit Erlangen-Nürnberg. thomas.koehler@fau.de
authors: Thomas
authors: T
authors: Köhler
authors: Pattern Recognition Lab, Friedrich-Alexander-Universitäit Erlangen-Nürnberg.
authors: Sven
authors: S
authors: Haase
authors: Pattern Recognition Lab, Friedrich-Alexander-Universitäit Erlangen-Nürnberg.
authors: Sebastian
authors: S
authors: Bauer
authors: Pattern Recognition Lab, Friedrich-Alexander-Universitäit Erlangen-Nürnberg.
authors: Jakob
authors: J
authors: Wasza
authors: Div. Medical and Biological Informatics Junior Group: Computer-Assisted, Interventions, German Cancer Research Center (DKFZ) Heidelberg.
authors: Thomas
authors: T
authors: Kilgus
authors: Div. Medical and Biological Informatics Junior Group: Computer-Assisted, Interventions, German Cancer Research Center (DKFZ) Heidelberg.
authors: Lena
authors: L
authors: Maier-Hein
authors: Minimally Invasive Therapy and Intervention, Technical University of Munich.
authors: Hubertus
authors: H
authors: Feubner
authors: Pattern Recognition Lab, Friedrich-Alexander-Universitäit Erlangen-Nürnberg.
authors: Joachim
authors: J
authors: Hornegger
mesh_terms: D000465:Algorithms; D003116:Color; D003124:Colorimetry / Q000295:instrumentation*; D004724:Endoscopy; D004867:Equipment Design; D019544:Equipment Failure Analysis; D007089:Image Enhancement / Q000295:instrumentation / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000295:instrumentation* / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted / Q000295:instrumentation*; D018511:Systems Integration; D014159:Transducers