title: The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech.
abstract: One of the key features of human interpersonal communication is our ability to integrate information communicated by speech and accompanying gestures. However, it is still not fully understood how this essential combinatory process is represented in the human brain. Functional magnetic resonance imaging (fMRI) studies have unanimously attested the relevance of activation in the posterior superior temporal sulcus/middle temporal gyrus (pSTS/MTG), while electroencephalography (EEG) studies have shown oscillatory activity in specific frequency bands to be associated with multisensory integration. In the current study, we used fMRI and EEG to separately investigate the anatomical and oscillatory neural signature of integrating intrinsically meaningful gestures (IMG; e.g. "Thumbs-up gesture") and corresponding speech (e.g., "The actor did a good job"). In both the fMRI (n =2 0) and EEG (n = 20) study, participants were presented with videos of an actor either: performing IMG in the context of a German sentence (GG), IMG in the context of a Russian (as a foreign language) sentence (GR), or speaking an isolated German sentence without gesture (SG). The results of the fMRI experiment confirmed that gesture-speech processing of IMG activates the posterior MTG (GG>GR∩GG>SG). In the EEG experiment we found that the identical integration process (GG>GR∩GG>SG) is related to a centrally-distributed alpha (7-13 Hz) power decrease within 700-1400 ms post-onset of the critical word. These new findings suggest that BOLD response increase in the pMTG and alpha power decrease represent the neural correlates of integrating intrinsically meaningful gestures with their corresponding speech.
authors.affiliation: Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Rudolf-Bultmann-Street 8, 35039 Marburg, Germany; Department of General Linguistics, Johannes Gutenberg-University Mainz, Jakob-Welder-Weg. 18, 55122 Mainz, Germany. Electronic address: yifei.he@staff.uni-marburg.de.
authors.forename: Yifei
authors.initials: Y
authors.lastname: He
authors.affiliation: Cognitive Neuroscience at Centre for Psychiatry, Justus Liebig University Giessen, Am Steg 28, 35385 Giessen, Germany.
authors.forename: Helge
authors.initials: H
authors.lastname: Gebhardt
authors.affiliation: Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Rudolf-Bultmann-Street 8, 35039 Marburg, Germany.
authors.forename: Miriam
authors.initials: M
authors.lastname: Steines
authors.affiliation: Cognitive Neuroscience at Centre for Psychiatry, Justus Liebig University Giessen, Am Steg 28, 35385 Giessen, Germany.
authors.forename: Gebhard
authors.initials: G
authors.lastname: Sammer
authors.affiliation: Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Rudolf-Bultmann-Street 8, 35039 Marburg, Germany.
authors.forename: Tilo
authors.initials: T
authors.lastname: Kircher
authors.affiliation: Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Rudolf-Bultmann-Street 8, 35039 Marburg, Germany.
authors.forename: Arne
authors.initials: A
authors.lastname: Nagels
authors.affiliation: Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Rudolf-Bultmann-Street 8, 35039 Marburg, Germany.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Straube
mesh_terms: D000328:Adult; D001921:Brain / Q000098:blood supply* / Q000502:physiology*; D001931:Brain Mapping; D032882:Comprehension; D004569:Electroencephalography; D005260:Female; D007839:Functional Laterality; D005868:Gestures; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007802:Language; D008279:Magnetic Resonance Imaging; D008297:Male; D010100:Oxygen / Q000097:blood; D010775:Photic Stimulation; D011930:Reaction Time; D013060:Speech / Q000502:physiology*; D055815:Young Adult
chemical_list: D010100:Oxygen
keywords: EEG; Gesture; Multisensory-integration; Oscillation; Speech; fMRI