title: Neural correlates of bimodal speech and gesture comprehension.
abstract: The present study examined the neural correlates of speech and hand gesture comprehension in a naturalistic context. Fifteen participants watched audiovisual segments of speech and gesture while event-related potentials (ERPs) were recorded to the speech. Gesture influenced the ERPs to the speech. Specifically, there was a right-lateralized N400 effect-reflecting semantic integration-when gestures mismatched versus matched the speech. In addition, early sensory components in bilateral occipital and frontal sites differentiated speech accompanied by matching versus non-matching gestures. These results suggest that hand gestures may be integrated with speech at early and late stages of language processing.
authors.affiliation: Psychology Department, Colgate University, Hamilton, NY 13346-1338, USA. skelly@mail.colgate.edu
authors.forename: Spencer D
authors.initials: SD
authors.lastname: Kelly
authors.forename: Corinne
authors.initials: C
authors.lastname: Kravitz
authors.forename: Michael
authors.initials: M
authors.lastname: Hopkins
mesh_terms: D000328:Adult; D001288:Attention / Q000502:physiology*; D002540:Cerebral Cortex / Q000502:physiology; D032882:Comprehension / Q000502:physiology*; D004569:Electroencephalography; D005071:Evoked Potentials / Q000502:physiology; D005260:Female; D005868:Gestures; D006801:Humans; D008297:Male; D011578:Psycholinguistics; D012660:Semantics; D013067:Speech Perception / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*