title: BodyWindows: enhancing a mannequin with projective augmented reality for exploring anatomy, physiology and medical procedures.
abstract: Augmented reality offers the potential to radically extend and enhance the capabilities of physical medical simulators such as full-body mannequin trainers. We have developed a system that transforms the surface of a mannequin simulator into both a display screen and an input device. The BodyWindows system enables a user to open, size, and reposition multiple viewports onto the simulator body. We demonstrate a dynamic viewport that displays a beating heart. Similar viewports could be used to display real-time physiological responses to interventions the user applies to the mannequin, such as injection of a simulated drug. Viewport windows can be overlapping and show anatomy at different depths, creating the illusion of "cutting" multiple windows into the body to reveal structures at different depths from the surface. The developed low-cost interface employees an IR light pen and the Nintendo Wiimote. We also report experiments using the Microsoft Kinect computer vision sensor to provide a completely hand-gesture based interface.
authors.affiliation: Simulation and Medical Technology R&D Center, University of Pittsburgh, Pittsburgh, PA, USA. jts35@pitt.edu
authors.forename: Joseph T
authors.initials: JT
authors.lastname: Samosky
authors.forename: Bo
authors.initials: B
authors.lastname: Wang
authors.forename: Douglas A
authors.initials: DA
authors.lastname: Nelson
authors.forename: Russell
authors.initials: R
authors.lastname: Bregman
authors.forename: Andrew
authors.initials: A
authors.lastname: Hosmer
authors.forename: Robert A
authors.initials: RA
authors.lastname: Weaver
mesh_terms: D003198:Computer Simulation; D003626:Data Display; D021621:Imaging, Three-Dimensional; D008348:Manikins; D008953:Models, Anatomic; D010829:Physiological Phenomena; D014584:User-Computer Interface