title: A Kinect-Based Sign Language Hand Gesture Recognition System for Hearing- and Speech-Impaired: A Pilot Study of Pakistani Sign Language.
abstract: Sign language provides hearing and speech impaired individuals with an interface to communicate with other members of the society. Unfortunately, sign language is not understood by most of the common people. For this, a gadget based on image processing and pattern recognition can provide with a vital aid for detecting and translating sign language into a vocal language. This work presents a system for detecting and understanding the sign language gestures by a custom built software tool and later translating the gesture into a vocal language. For the purpose of recognizing a particular gesture, the system employs a Dynamic Time Warping (DTW) algorithm and an off-the-shelf software tool is employed for vocal language generation. Microsoft(®) Kinect is the primary tool used to capture video stream of a user. The proposed method is capable of successfully detecting gestures stored in the dictionary with an accuracy of 91%. The proposed system has the ability to define and add custom made gestures. Based on an experiment in which 10 individuals with impairments used the system to communicate with 5 people with no disability, 87% agreed that the system was useful.
authors.affiliation: a Faculty of Computer Science and Engineering , Ghulam Ishaq Khan Institute of Engineering Sciences and Technology , Topi , Pakistan.
authors.forename: Zahid
authors.initials: Z
authors.lastname: Halim
authors.forename: Ghulam
authors.initials: G
authors.lastname: Abbas
mesh_terms: D061725:Accelerometry / Q000295:instrumentation; D003143:Communication Devices for People with Disabilities; D012048:Correction of Hearing Impairment / Q000295:instrumentation* / Q000379:methods; D004867:Equipment Design; D019544:Equipment Failure Analysis; D005260:Female; D005868:Gestures; D006225:Hand / Q000502:physiology; D006311:Hearing Disorders; D006801:Humans; D008297:Male; D063731:Mobile Applications; D009068:Movement / Q000502:physiology; D010154:Pakistan; D010363:Pattern Recognition, Automated / Q000379:methods; D010865:Pilot Projects; D012813:Sign Language; D013064:Speech Disorders / Q000534:rehabilitation*; D014175:Translating; D016896:Treatment Outcome; D014584:User-Computer Interface; D018910:Video Games; D014741:Video Recording; D055815:Young Adult
keywords: communication aids for disabled; human-computer interface; image recognition; image sequence analysis; multimodal sensors; pattern recognition; sign language recognition