title: Virtually transparent surgical instruments in endoscopic surgery with augmentation of obscured regions.
abstract: OBJECTIVE
  We developed and evaluated a visual compensation system that allows surgeons to visualize obscured regions in real time, such that the surgical instrument appears virtually transparent.
  
  
  METHODS
  The system consists of two endoscopes: a main endoscope to observe the surgical environment, and a supporting endoscope to render the region hidden from view by surgical instruments. The view captured by the supporting endoscope is transformed to simulate the view from the main endoscope, segmented to the shape of the hidden regions, and superimposed to the main endoscope image so that the surgical instruments look transparent. A prototype device was benchmarked for processing time and superimposition rendering error. Then, it was evaluated in a training environment with 22 participants performing a backhand needle driving task with needle exit point error as the criterion. Lastly, we conducted an in vivo study.
  
  
  RESULTS
  In the benchmark, the mean processing time was 62.4 ms, which was lower than the processing time accepted in remote surgeries. The mean superimposition error of the superimposed image was 1.4 mm. In the training environment, needle exit point error with the system decreased significantly for experts compared with the condition without the system. This change was not significant for novices. In the in vivo study, our prototype enabled visualization of needle exit points during anastomosis.
  
  
  CONCLUSIONS
  The benchmark suggests that the implemented system had an acceptable performance, and evaluation in the training environment demonstrated improved surgical task outcomes in expert surgeons. We will conduct a more comprehensive in vivo study in the future.
authors.affiliation: Graduate School of Science and Engineering, Waseda University, 59-309, 3-4-1, Okubo, Shinjuku Ward, Tokyo, Japan. yuta-is-here@ruri.waseda.jp.|Center for Exploratory Research, Hitachi, Ltd., Tokyo, Japan. yuta-is-here@ruri.waseda.jp.
authors.forename: Yuta
authors.initials: Y
authors.lastname: Koreeda
authors.affiliation: Faculty of Science and Engineering in the Graduate School of Science and Engineering, Waseda University, Tokyo, Japan.
authors.forename: Yo
authors.initials: Y
authors.lastname: Kobayashi
authors.affiliation: Center for the Integration of Advanced Medicine and Innovative Technology, Kyushu University, Fukuoka, Japan.
authors.forename: Satoshi
authors.initials: S
authors.lastname: Ieiri
authors.affiliation: Graduate School of Science and Engineering, Waseda University, 59-309, 3-4-1, Okubo, Shinjuku Ward, Tokyo, Japan.
authors.forename: Yuya
authors.initials: Y
authors.lastname: Nishio
authors.affiliation: Center for Frontier Medical Engineering, Chiba University, Chiba, Japan.
authors.forename: Kazuya
authors.initials: K
authors.lastname: Kawamura
authors.affiliation: Center for the Integration of Advanced Medicine and Innovative Technology, Kyushu University, Fukuoka, Japan.
authors.forename: Satoshi
authors.initials: S
authors.lastname: Obata
authors.affiliation: Center for the Integration of Advanced Medicine and Innovative Technology, Kyushu University, Fukuoka, Japan.
authors.forename: Ryota
authors.initials: R
authors.lastname: Souzaki
authors.affiliation: Center for the Integration of Advanced Medicine and Innovative Technology, Kyushu University, Fukuoka, Japan.
authors.forename: Makoto
authors.initials: M
authors.lastname: Hashizume
authors.affiliation: Faculty of Science and Engineering in the Graduate School of Science and Engineering, Waseda University, Tokyo, Japan.
authors.forename: Masakatsu G
authors.initials: MG
authors.lastname: Fujie
mesh_terms: D000818:Animals; D019723:Endoscopes; D004724:Endoscopy / Q000379:methods*; D004867:Equipment Design; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D025321:Surgery, Computer-Assisted / Q000379:methods*; D013525:Surgical Instruments; D013536:Suture Techniques; D013552:Swine; D014584:User-Computer Interface
keywords: Augmented reality; Computer-assisted surgery; Endoscopic surgery; Laparoscopic surgery; Medical image processing; Visualization