Real and virtual explorations of the environment and interactive tracking of movable objects for the blind on the basis of tactile-acoustical maps and 3D environment models.
PURPOSE.: This study describes the development of a multi-functional assistant system for the blind which combines localisation, real and virtual navigation within modelled environments and the identification and tracking of fixed and movable objects.


METHODS
The approximate position of buildings is determined with a global positioning sensor (GPS), then the user establishes exact position at a specific landmark, like a door. This location initialises indoor navigation, based on an inertial sensor, a step recognition algorithm and map. Tracking of movable objects is provided by another inertial sensor and a head-mounted stereo camera, combined with 3D environmental models. This study developed an algorithm based on shape and colour to identify objects and used a common face detection algorithm to inform the user of the presence and position of others.


RESULTS
The system allows blind people to determine their position with approximately 1 metre accuracy. Virtual exploration of the environment can be accomplished by moving one's finger on a touch screen of a small portable tablet PC. The name of rooms, building features and hazards, modelled objects and their positions are presented acoustically or in Braille.


CONCLUSIONS
Given adequate environmental models, this system offers blind people the opportunity to navigate independently and safely, even within unknown environments. Additionally, the system facilitates education and rehabilitation by providing, in several languages, object names, features and relative positions.
Hub
Andreas
A
Visualization and Interactive Systems Group, University of Stuttgart, Stuttgart, Germany. hub@vis.uni-stuttgart.de
Hartter
Tim
T
Kombrink
Stefan
S
Ertl
Thomas
T
D000162:Acoustics / Q000295:instrumentation*; D000465:Algorithms; D001766:Blindness; D016287:Computer Peripherals; D034201:Computers, Handheld; D006233:Persons with Disabilities / Q000534:rehabilitation*; D004777:Environment; D006801:Humans; D021621:Imaging, Three-Dimensional; D012656:Self-Help Devices; D012984:Software; D014110:Touch / Q000502:physiology*; D014584:User-Computer Interface