title: Dex-ray: augmented reality neurosurgical navigation with a handheld video probe.
abstract: OBJECTIVE
  We developed an augmented reality system that enables intraoperative image guidance by using 3-dimensional (3D) graphics overlaid on a video stream. We call this system DEX-Ray and report on its development and the initial intraoperative experience in 12 cases.
  
  
  METHODS
  DEX-Ray consists of a tracked handheld probe that integrates a lipstick-size video camera. The camera looks over the probe's tip into the surgical field. The camera's video stream is augmented with coregistered, multimodality 3D graphics and landmarks obtained during neurosurgical planning with 3D workstations. The handheld probe functions as a navigation device to view and point and as an interaction device to adjust the 3D graphics. We tested the system's accuracy in the laboratory and evaluated it intraoperatively with a series of tumor and vascular cases.
  
  
  RESULTS
  DEX-Ray provided accurate and real-time video-based augmented reality display. The system could be seamlessly integrated into the surgical workflow. The see-through effect revealing 3D information below the surgically exposed surface proved to be of significant value, especially during the macroscopic phase of an operation, providing easily understandable structural navigational information. Navigation in deep and narrow surgical corridors was limited by the camera resolution and light sensitivity.
  
  
  CONCLUSIONS
  The system was perceived as an improved navigational experience because the augmented see-through effect allowed direct understanding of the surgical anatomy beyond the visible surface and direct guidance toward surgical targets.
authors.affiliation: Department of Neurosurgery, University Hospital Zürich, Zürich, Switzerland. RalfAlfons.kockro@usz.ch
authors.forename: Ralf A
authors.initials: RA
authors.lastname: Kockro
authors.forename: Yeo Tseng
authors.initials: YT
authors.lastname: Tsai
authors.forename: Ivan
authors.initials: I
authors.lastname: Ng
authors.forename: Peter
authors.initials: P
authors.lastname: Hwang
authors.forename: Chuangui
authors.initials: C
authors.lastname: Zhu
authors.forename: Kusuma
authors.initials: K
authors.lastname: Agusanto
authors.forename: Liang Xiao
authors.initials: LX
authors.lastname: Hong
authors.forename: Luis
authors.initials: L
authors.lastname: Serra
mesh_terms: D000328:Adult; D000368:Aged; D001921:Brain / Q000033:anatomy & histology / Q000601:surgery; D003201:Computers / Q000639:trends; D004867:Equipment Design / Q000379:methods; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000295:instrumentation / Q000379:methods; D008279:Magnetic Resonance Imaging; D008875:Middle Aged; D038361:Neuronavigation / Q000295:instrumentation* / Q000379:methods; D019635:Neurosurgical Procedures / Q000295:instrumentation* / Q000379:methods; D009949:Orientation; D011300:Preoperative Care / Q000295:instrumentation / Q000379:methods; D013028:Space Perception / Q000502:physiology; D013525:Surgical Instruments / Q000639:trends; D013663:Teaching; D014057:Tomography, X-Ray Computed; D014584:User-Computer Interface; D014741:Video Recording / Q000295:instrumentation* / Q000379:methods