Effects of lips and hands on auditory learning of second-language speech sounds.
OBJECTIVE
Previous research has found that auditory training helps native English speakers to perceive phonemic vowel length contrasts in Japanese, but their performance did not reach native levels after training. Given that multimodal information, such as lip movement and hand gesture, influences many aspects of native language processing, the authors examined whether multimodal input helps to improve native English speakers' ability to perceive Japanese vowel length contrasts.


METHODS
Sixty native English speakers participated in 1 of 4 types of training: (a) audio-only; (b) audio-mouth; (c) audio-hands; and (d) audio-mouth-hands. Before and after training, participants were given phoneme perception tests that measured their ability to identify short and long vowels in Japanese (e.g., /kato/ vs. /kato/).


RESULTS
Although all 4 groups improved from pre- to posttest (replicating previous research), the participants in the audio-mouth condition improved more than those in the audio-only condition, whereas the 2 conditions involving hand gestures did not.


CONCLUSIONS
Seeing lip movements during training significantly helps learners to perceive difficult second-language phonemic contrasts, but seeing hand gestures does not. The authors discuss possible benefits and limitations of using multimodal information in second-language phoneme learning.
Hirata
Yukari
Y
Colgate University, Hamilton, NY, USA. yhirata@mail.colgate.edu
Kelly
Spencer D
SD
D000704:Analysis of Variance; D001307:Auditory Perception; D005868:Gestures; D006225:Hand; D006801:Humans; D007802:Language; D007807:Language Tests; D007858:Learning; D008046:Lip; D008085:Lipreading; D009043:Motor Activity; D019303:Multilingualism; D010700:Phonetics; D011578:Psycholinguistics; D013060:Speech; D014796:Visual Perception; D055815:Young Adult