Two sides of the same coin: speech and gesture mutually interact to enhance comprehension.
Gesture and speech are assumed to form an integrated system during language production. Based on this view, we propose the integrated-systems hypothesis, which explains two ways in which gesture and speech are integrated--through mutual and obligatory interactions--in language comprehension. Experiment 1 presented participants with action primes (e.g., someone chopping vegetables) and bimodal speech and gesture targets. Participants related primes to targets more quickly and accurately when they contained congruent information (speech: "chop"; gesture: chop) than when they contained incongruent information (speech: "chop"; gesture: twist). Moreover, the strength of the incongruence affected processing, with fewer errors for weak incongruities (speech: "chop"; gesture: cut) than for strong incongruities (speech: "chop"; gesture: twist). Crucial for the integrated-systems hypothesis, this influence was bidirectional. Experiment 2 demonstrated that gesture's influence on speech was obligatory. The results confirm the integrated-systems hypothesis and demonstrate that gesture and speech form an integrated system in language comprehension.
Kelly
Spencer D
SD
Department of Psychology, Colgate University, Hamilton, NY 13346, USA. skelly@colgate.edu
Ozyürek
Asli
A
Maris
Eric
E
D001288:Attention; D032882:Comprehension; D005260:Female; D005868:Gestures; D006801:Humans; D008297:Male; D012660:Semantics; D013060:Speech