title: Multi-time resolution analysis of speech: evidence from psychophysics.
abstract: How speech signals are analyzed and represented remains a foundational challenge both for cognitive science and neuroscience. A growing body of research, employing various behavioral and neurobiological experimental techniques, now points to the perceptual relevance of both phoneme-sized (10-40 Hz modulation frequency) and syllable-sized (2-10 Hz modulation frequency) units in speech processing. However, it is not clear how information associated with such different time scales interacts in a manner relevant for speech perception. We report behavioral experiments on speech intelligibility employing a stimulus that allows us to investigate how distinct temporal modulations in speech are treated separately and whether they are combined. We created sentences in which the slow (~4 Hz; Slow) and rapid (~33 Hz; Shigh) modulations-corresponding to ~250 and ~30 ms, the average duration of syllables and certain phonetic properties, respectively-were selectively extracted. Although Slow and Shigh have low intelligibility when presented separately, dichotic presentation of Shigh with Slow results in supra-additive performance, suggesting a synergistic relationship between low- and high-modulation frequencies. A second experiment desynchronized presentation of the Slow and Shigh signals. Desynchronizing signals relative to one another had no impact on intelligibility when delays were less than ~45 ms. Longer delays resulted in a steep intelligibility decline, providing further evidence of integration or binding of information within restricted temporal windows. Our data suggest that human speech perception uses multi-time resolution processing. Signals are concurrently analyzed on at least two separate time scales, the intermediate representations of these analyses are integrated, and the resulting bound percept has significant consequences for speech intelligibility-a view compatible with recent insights from neuroscience implicating multi-timescale auditory processing.
authors.affiliation: Neuroscience and Cognitive Science Program, University of Maryland College Park, MD, USA ; Department of Linguistics, University of Maryland College Park, MD, USA.
authors.forename: Maria
authors.initials: M
authors.lastname: Chait
authors.affiliation: Silicon Speech Hidden Valley Lake, CA, USA.
authors.forename: Steven
authors.initials: S
authors.lastname: Greenberg
authors.affiliation: Department of Information and Communication Sciences, Sophia University Tokyo, Japan.
authors.forename: Takayuki
authors.initials: T
authors.lastname: Arai
authors.affiliation: Neuroscience and Cognitive Science Program, University of Maryland College Park, MD, USA ; Department of Biology, University of Maryland College Park, MD, USA ; Department of Electrical and Computer Engineering, University of Maryland College Park, MD, USA ; Institute for Systems Research, University of Maryland College Park, MD, USA.
authors.forename: Jonathan Z
authors.initials: JZ
authors.lastname: Simon
authors.affiliation: Neuroscience and Cognitive Science Program, University of Maryland College Park, MD, USA ; Department of Linguistics, University of Maryland College Park, MD, USA ; Department of Psychology, New York University New York, NY, USA ; Department of Neuroscience, Max-Planck-Institute Frankfurt, Germany.
authors.forename: David
authors.initials: D
authors.lastname: Poeppel
keywords: auditory processing; modulation spectrum; phoneme; speech perception; speech segmentation; syllable; temporal processing