Dex-ray: augmented reality neurosurgical navigation with a handheld video probe.
OBJECTIVE
We developed an augmented reality system that enables intraoperative image guidance by using 3-dimensional (3D) graphics overlaid on a video stream. We call this system DEX-Ray and report on its development and the initial intraoperative experience in 12 cases.


METHODS
DEX-Ray consists of a tracked handheld probe that integrates a lipstick-size video camera. The camera looks over the probe's tip into the surgical field. The camera's video stream is augmented with coregistered, multimodality 3D graphics and landmarks obtained during neurosurgical planning with 3D workstations. The handheld probe functions as a navigation device to view and point and as an interaction device to adjust the 3D graphics. We tested the system's accuracy in the laboratory and evaluated it intraoperatively with a series of tumor and vascular cases.


RESULTS
DEX-Ray provided accurate and real-time video-based augmented reality display. The system could be seamlessly integrated into the surgical workflow. The see-through effect revealing 3D information below the surgically exposed surface proved to be of significant value, especially during the macroscopic phase of an operation, providing easily understandable structural navigational information. Navigation in deep and narrow surgical corridors was limited by the camera resolution and light sensitivity.


CONCLUSIONS
The system was perceived as an improved navigational experience because the augmented see-through effect allowed direct understanding of the surgical anatomy beyond the visible surface and direct guidance toward surgical targets.
Kockro
Ralf A
RA
Department of Neurosurgery, University Hospital Zürich, Zürich, Switzerland. RalfAlfons.kockro@usz.ch
Tsai
Yeo Tseng
YT
Ng
Ivan
I
Hwang
Peter
P
Zhu
Chuangui
C
Agusanto
Kusuma
K
Hong
Liang Xiao
LX
Serra
Luis
L
D000328:Adult; D000368:Aged; D001921:Brain / Q000033:anatomy & histology / Q000601:surgery; D003201:Computers / Q000639:trends; D004867:Equipment Design / Q000379:methods; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000295:instrumentation / Q000379:methods; D008279:Magnetic Resonance Imaging; D008875:Middle Aged; D038361:Neuronavigation / Q000295:instrumentation* / Q000379:methods; D019635:Neurosurgical Procedures / Q000295:instrumentation* / Q000379:methods; D009949:Orientation; D011300:Preoperative Care / Q000295:instrumentation / Q000379:methods; D013028:Space Perception / Q000502:physiology; D013525:Surgical Instruments / Q000639:trends; D013663:Teaching; D014057:Tomography, X-Ray Computed; D014584:User-Computer Interface; D014741:Video Recording / Q000295:instrumentation* / Q000379:methods