title: Robust augmented reality registration method for localization of solid organs' tumors using CT-derived virtual biomechanical model and fluorescent fiducials.
abstract: Augmented reality (AR) is the fusion of computer-generated and real-time images. AR can be used in surgery as a navigation tool, by creating a patient-specific virtual model through 3D software manipulation of DICOM imaging (e.g., CT scan). The virtual model can be superimposed to real-time images enabling transparency visualization of internal anatomy and accurate localization of tumors. However, the 3D model is rigid and does not take into account inner structures' deformations. We present a concept of automated AR registration, while the organs undergo deformation during surgical manipulation, based on finite element modeling (FEM) coupled with optical imaging of fluorescent surface fiducials.
  
  
  
  Two 10 × 1 mm wires (pseudo-tumors) and six 10 × 0.9 mm fluorescent fiducials were placed in ex vivo porcine kidneys (n = 10). Biomechanical FEM-based models were generated from CT scan. Kidneys were deformed and the shape changes were identified by tracking the fiducials, using a near-infrared optical system. The changes were registered automatically with the virtual model, which was deformed accordingly. Accuracy of prediction of pseudo-tumors' location was evaluated with a CT scan in the deformed status (ground truth). In vivo: fluorescent fiducials were inserted under ultrasound guidance in the kidney of one pig, followed by a CT scan. The FEM-based virtual model was superimposed on laparoscopic images by automatic registration of the fiducials.
  
  
  
  Biomechanical models were successfully generated and accurately superimposed on optical images. The mean measured distance between the estimated tumor by biomechanical propagation and the scanned tumor (ground truth) was 0.84 ± 0.42 mm. All fiducials were successfully placed in in vivo kidney and well visualized in near-infrared mode enabling accurate automatic registration of the virtual model on the laparoscopic images.
  
  
  
  Our preliminary experiments showed the potential of a biomechanical model with fluorescent fiducials to propagate the deformation of solid organs' surface to their inner structures including tumors with good accuracy and automatized robust tracking.
authors: IHU-Strasbourg, Institute of Image-Guided Surgery, Strasbourg, France.|Department of Surgery, Seoul National University Hospital, Seoul, Korea.
authors: Seong-Ho
authors: SH
authors: Kong
authors: Institut national de recherche en informatique et en automatique (INRIA) Mimesis, Strasbourg, France.
authors: Nazim
authors: N
authors: Haouchine
authors: IHU-Strasbourg, Institute of Image-Guided Surgery, Strasbourg, France.
authors: Renato
authors: R
authors: Soares
authors: Biophotonic and Pharmacology Lab, UMR 7213 CNRS, Pharmacological Faculty, University of Strasbourg, Strasbourg, France.
authors: Andrey
authors: A
authors: Klymchenko
authors: Biophotonic and Pharmacology Lab, UMR 7213 CNRS, Pharmacological Faculty, University of Strasbourg, Strasbourg, France.
authors: Bohdan
authors: B
authors: Andreiuk
authors: Institut national de recherche en informatique et en automatique (INRIA) Mimesis, Strasbourg, France.
authors: Bruno
authors: B
authors: Marques
authors: IRCAD, Research Institute against Cancer of the Digestive System, 1, Place de l'Hôpital, 67091, Strasbourg, France.
authors: Galyna
authors: G
authors: Shabat
authors: Division of Urology, Clinique Saint-Augustin, Bordeaux, France.
authors: Thierry
authors: T
authors: Piechaud
authors: IHU-Strasbourg, Institute of Image-Guided Surgery, Strasbourg, France. michele.diana@ircad.fr.|IRCAD, Research Institute against Cancer of the Digestive System, 1, Place de l'Hôpital, 67091, Strasbourg, France. michele.diana@ircad.fr.
authors: Michele
authors: 0000-0002-1390-8486
authors: M
authors: Diana
authors: Institut national de recherche en informatique et en automatique (INRIA) Mimesis, Strasbourg, France.
authors: Stéphane
authors: S
authors: Cotin
authors: IHU-Strasbourg, Institute of Image-Guided Surgery, Strasbourg, France.|IRCAD, Research Institute against Cancer of the Digestive System, 1, Place de l'Hôpital, 67091, Strasbourg, France.
authors: Jacques
authors: J
authors: Marescaux
mesh_terms: D000818:Animals; D001696:Biomechanical Phenomena; D057918:Fiducial Markers; D020342:Finite Element Analysis; D005456:Fluorescent Dyes; D021621:Imaging, Three-Dimensional / Q000379:methods*; D066298:In Vitro Techniques; D007668:Kidney / Q000000981:diagnostic imaging / Q000601:surgery*; D010535:Laparoscopy; D008953:Models, Anatomic; D009369:Neoplasms / Q000000981:diagnostic imaging / Q000601:surgery; D025321:Surgery, Computer-Assisted / Q000379:methods*; D013552:Swine; D014057:Tomography, X-Ray Computed; D000076142:Virtual Reality
chemical_list: D005456:Fluorescent Dyes
keywords: Augmented reality; Automatic registration; Fiducials; Finite element modeling; Fluorescence-guided surgery; Optical imaging; Solid organ tumor