title: "Look who's talking!" Gaze Patterns for Implicit and Explicit Audio-Visual Speech Synchrony Detection in Children With High-Functioning Autism.
abstract: Conversation requires integration of information from faces and voices to fully understand the speaker's message. To detect auditory-visual asynchrony of speech, listeners must integrate visual movements of the face, particularly the mouth, with auditory speech information. Individuals with autism spectrum disorder may be less successful at such multisensory integration, despite their demonstrated preference for looking at the mouth region of a speaker. We showed participants (individuals with and without high-functioning autism (HFA) aged 8-19) a split-screen video of two identical individuals speaking side by side. Only one of the speakers was in synchrony with the corresponding audio track and synchrony switched between the two speakers every few seconds. Participants were asked to watch the video without further instructions (implicit condition) or to specifically watch the in-synch speaker (explicit condition). We recorded which part of the screen and face their eyes targeted. Both groups looked at the in-synch video significantly more with explicit instructions. However, participants with HFA looked at the in-synch video less than typically developing (TD) peers and did not increase their gaze time as much as TD participants in the explicit task. Importantly, the HFA group looked significantly less at the mouth than their TD peers, and significantly more at non-face regions of the image. There were no between-group differences for eye-directed gaze. Overall, individuals with HFA spend less time looking at the crucially important mouth region of the face during auditory-visual speech integration, which is maladaptive gaze behavior for this type of task.
authors.affiliation: Emerson College, Department of Communication Sciences and Disorders, 120 Boylston Street, Boston, Massachusetts.|University of Massachusetts Medical School Shriver Center, 200 Trapelo Rd, Waltham, Massachusetts.
authors.forename: Ruth B
authors.initials: RB
authors.lastname: Grossman
authors.affiliation: University of Massachusetts Medical School Shriver Center, 200 Trapelo Rd, Waltham, Massachusetts.
authors.forename: Erin
authors.initials: E
authors.lastname: Steinhart
authors.affiliation: University of Massachusetts Medical School Shriver Center, 200 Trapelo Rd, Waltham, Massachusetts.
authors.forename: Teresa
authors.initials: T
authors.lastname: Mitchell
authors.affiliation: University of Massachusetts Medical School Shriver Center, 200 Trapelo Rd, Waltham, Massachusetts.
authors.forename: William
authors.initials: W
authors.lastname: McIlvane
mesh_terms: D000161:Acoustic Stimulation; D000293:Adolescent; D000328:Adult; D001288:Attention / Q000502:physiology; D001307:Auditory Perception / Q000502:physiology*; D001321:Autistic Disorder / Q000503:physiopathology*; D002648:Child; D005133:Eye Movements / Q000502:physiology*; D005145:Face; D005260:Female; D005403:Fixation, Ocular / Q000502:physiology; D006801:Humans; D008297:Male; D010775:Photic Stimulation; D013060:Speech / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult
keywords: audio-visual integration; eye tracking; face perception; high-functioning autism; mouth-directed gaze