Neural integration of iconic and unrelated coverbal gestures: a functional MRI study.
Gestures are an important part of interpersonal communication, for example by illustrating physical properties of speech contents (e.g., "the ball is round"). The meaning of these so-called iconic gestures is strongly intertwined with speech. We investigated the neural correlates of the semantic integration for verbal and gestural information. Participants watched short videos of five speech and gesture conditions performed by an actor, including variation of language (familiar German vs. unfamiliar Russian), variation of gesture (iconic vs. unrelated), as well as isolated familiar language, while brain activation was measured using functional magnetic resonance imaging. For familiar speech with either of both gesture types contrasted to Russian speech-gesture pairs, activation increases were observed at the left temporo-occipital junction. Apart from this shared location, speech with iconic gestures exclusively engaged left occipital areas, whereas speech with unrelated gestures activated bilateral parietal and posterior temporal regions. Our results demonstrate that the processing of speech with speech-related versus speech-unrelated gestures occurs in two distinct but partly overlapping networks. The distinct processing streams (visual versus linguistic/spatial) are interpreted in terms of "auxiliary systems" allowing the integration of speech and gesture in the left temporo-occipital region.
Department of Psychiatry and Psychotherapy-Section of Experimental Psychopathology, RWTH Aachen University, Aachen, Germany. psychiat@med.uni-marburg.de
Antonia
A
Green
Benjamin
B
Straube
Susanne
S
Weis
Andreas
A
Jansen
Klaus
K
Willmes
Kerstin
K
Konrad
Tilo
T
Kircher
D000328:Adult; D000704:Analysis of Variance; D001921:Brain / Q000098:blood supply / Q000502:physiology*; D001931:Brain Mapping; D032882:Comprehension / Q000502:physiology*; D005868:Gestures; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods; D008279:Magnetic Resonance Imaging / Q000379:methods; D008297:Male; D008875:Middle Aged; D010100:Oxygen / Q000097:blood; D010775:Photic Stimulation / Q000379:methods; D021641:Recognition, Psychology / Q000502:physiology*; D013060:Speech / Q000502:physiology*; D055815:Young Adult
D010100:Oxygen