title: Informatics in radiology: Intuitive user interface for 3D image manipulation using augmented reality and a smartphone as a remote control.
abstract: Although widely used as a pointing device on personal computers (PCs), the mouse was originally designed for control of two-dimensional (2D) cursor movement and is not suited to complex three-dimensional (3D) image manipulation. Augmented reality (AR) is a field of computer science that involves combining the physical world and an interactive 3D virtual world; it represents a new 3D user interface (UI) paradigm. A system for 3D and four-dimensional (4D) image manipulation has been developed that uses optical tracking AR integrated with a smartphone remote control. The smartphone is placed in a hard case (jacket) with a 2D printed fiducial marker for AR on the back. It is connected to a conventional PC with an embedded Web camera by means of WiFi. The touch screen UI of the smartphone is then used as a remote control for 3D and 4D image manipulation. Using this system, the radiologist can easily manipulate 3D and 4D images from computed tomography and magnetic resonance imaging in an AR environment with high-quality image resolution. Pilot assessment of this system suggests that radiologists will be able to manipulate 3D and 4D images in the reading room in the near future. Supplemental material available at http://radiographics.rsna.org/lookup/suppl/doi:10.1148/rg.324115086/-/DC1.
authors.affiliation: Department of Radiology and Institute for High Dimensional Medical Imaging, Research Center for Medical Sciences, Jikei University School of Medicine, 3-25-8 Nishi-Shinbashi, Minato-ku, Tokyo 1058461, Japan. nakata@jikei.ac.jp
authors.forename: Norio
authors.initials: N
authors.lastname: Nakata
authors.forename: Naoki
authors.initials: N
authors.lastname: Suzuki
authors.forename: Asaki
authors.initials: A
authors.lastname: Hattori
authors.forename: Naoya
authors.initials: N
authors.lastname: Hirai
authors.forename: Yukio
authors.initials: Y
authors.lastname: Miyamoto
authors.forename: Kunihiko
authors.initials: K
authors.lastname: Fukuda
mesh_terms: D040421:Cell Phone; D016287:Computer Peripherals; D034201:Computers, Handheld; D004867:Equipment Design; D019544:Equipment Failure Analysis; D007089:Image Enhancement / Q000295:instrumentation; D007090:Image Interpretation, Computer-Assisted / Q000295:instrumentation*; D021621:Imaging, Three-Dimensional / Q000295:instrumentation*; D008491:Medical Informatics Applications; D013686:Telemetry / Q000295:instrumentation; D014584:User-Computer Interface