title: Multisensory object perception in infancy: 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object.
abstract: Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants' use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8% mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.
authors.affiliation: Perceptual Development Laboratory, Boys Town National Research Hospital, 555 N. 30th Street, Omaha, NE 68131, United States.
authors.forename: Nicholas A
authors.initials: NA
authors.lastname: Smith
authors.affiliation: Department of Psychology, Neuroscience and Behaviour, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
authors.forename: Nicole A
authors.initials: NA
authors.lastname: Folland
authors.affiliation: Department of Psychology, Neuroscience and Behaviour, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
authors.forename: Diana M
authors.initials: DM
authors.lastname: Martinez
authors.affiliation: Department of Psychology, Neuroscience and Behaviour, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; McMaster Institute for Music and the Mind, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Rotman Research Institute, Baycrest, University of Toronto, 3560 Bathurst Street, Toronto, Ontario M6A 2E1, Canada. Electronic address: LJT@mcmaster.ca.
authors.forename: Laurel J
authors.initials: LJ
authors.lastname: Trainor
mesh_terms: D000161:Acoustic Stimulation; D001307:Auditory Perception / Q000502:physiology*; D005260:Female; D006801:Humans; D007223:Infant; D008297:Male; D010775:Photic Stimulation; D010898:Pitch Perception / Q000502:physiology; D014796:Visual Perception / Q000502:physiology*
keywords: Auditory development; Auditory scene analysis; Multisensory object perception; Simultaneous integration