title: Multimodal integration in statistical learning: evidence from the McGurk illusion.
abstract: Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker's face. In the critical condition, the visual (e.g., /gi/) and auditory (e.g., /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g., /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants' ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning.
authors.affiliation: Department of Psychology and Program in Neuroscience, Bucknell University Lewisburg, PA, USA.
authors.forename: Aaron D
authors.initials: AD
authors.lastname: Mitchel
authors.affiliation: Department of Psychology, Cornell University Ithaca, NY, USA ; Department of Language and Communication, University of Southern Denmark Odense, Denmark ; Haskins Laboratories, New Haven CT, USA.
authors.forename: Morten H
authors.initials: MH
authors.lastname: Christiansen
authors.affiliation: Department of Psychology and Program in Linguistics, Pennsylvania State University, University Park PA, USA.
authors.forename: Daniel J
authors.initials: DJ
authors.lastname: Weiss
keywords: McGurk illusion; audiovisual speech perception; language acquisition; multisensory integration; multisensory perception; multisensory statistical learning; statistical learning mechanisms