title: Language/Culture Modulates Brain and Gaze Processes in Audiovisual Speech Perception.
abstract: Several behavioural studies have shown that the interplay between voice and face information in audiovisual speech perception is not universal. Native English speakers (ESs) are influenced by visual mouth movement to a greater degree than native Japanese speakers (JSs) when listening to speech. However, the biological basis of these group differences is unknown. Here, we demonstrate the time-varying processes of group differences in terms of event-related brain potentials (ERP) and eye gaze for audiovisual and audio-only speech perception. On a behavioural level, while congruent mouth movement shortened the ESs' response time for speech perception, the opposite effect was observed in JSs. Eye-tracking data revealed a gaze bias to the mouth for the ESs but not the JSs, especially before the audio onset. Additionally, the ERP P2 amplitude indicated that ESs processed multisensory speech more efficiently than auditory-only speech; however, the JSs exhibited the opposite pattern. Taken together, the ESs' early visual attention to the mouth was likely to promote phonetic anticipation, which was not the case for the JSs. These results clearly indicate the impact of language and/or culture on multisensory speech processing, suggesting that linguistic/cultural experiences lead to the development of unique neural systems for audiovisual speech perception.
authors.affiliation: Division of Cognitive Psychology, Faculty of Letters, Kumamoto University 2-40-1, Kurokami, Chuo-ku, Kumamoto, 860-8555, Japan.
authors.forename: Satoko
authors.initials: S
authors.lastname: Hisanaga
authors.affiliation: Division of Cognitive Psychology, Faculty of Letters, Kumamoto University 2-40-1, Kurokami, Chuo-ku, Kumamoto, 860-8555, Japan.
authors.forename: Kaoru
authors.initials: K
authors.lastname: Sekiyama
authors.affiliation: Department of Information Technology on Human and Environmental Science, Graduate School of Science and Technology, Kumamoto University, 2-39-1, Kurokami, Chuo-ku, Kumamoto, 860-8555, Japan.
authors.forename: Tomohiko
authors.initials: T
authors.lastname: Igasaki
authors.affiliation: Department of Information Technology on Human and Environmental Science, Graduate School of Science and Technology, Kumamoto University, 2-39-1, Kurokami, Chuo-ku, Kumamoto, 860-8555, Japan.
authors.forename: Nobuki
authors.initials: N
authors.lastname: Murayama
mesh_terms: D000328:Adult; D001307:Auditory Perception; D001921:Brain / Q000502:physiology*; D003469:Culture; D005071:Evoked Potentials; D005260:Female; D006801:Humans; D007802:Language; D008297:Male; D013067:Speech Perception; D014796:Visual Perception; D055815:Young Adult