title: Co-speech iconic gestures and visuo-spatial working memory.
abstract: Three experiments tested the role of verbal versus visuo-spatial working memory in the comprehension of co-speech iconic gestures. In Experiment 1, participants viewed congruent discourse primes in which the speaker's gestures matched the information conveyed by his speech, and incongruent ones in which the semantic content of the speaker's gestures diverged from that in his speech. Discourse primes were followed by picture probes that participants judged as being either related or unrelated to the preceding clip. Performance on this picture probe classification task was faster and more accurate after congruent than incongruent discourse primes. The effect of discourse congruency on response times was linearly related to measures of visuo-spatial, but not verbal, working memory capacity, as participants with greater visuo-spatial WM capacity benefited more from congruent gestures. In Experiments 2 and 3, participants performed the same picture probe classification task under conditions of high and low loads on concurrent visuo-spatial (Experiment 2) and verbal (Experiment 3) memory tasks. Effects of discourse congruency and verbal WM load were additive, while effects of discourse congruency and visuo-spatial WM load were interactive. Results suggest that congruent co-speech gestures facilitate multi-modal language comprehension, and indicate an important role for visuo-spatial WM in these speech-gesture integration processes.
authors.affiliation: Center for Research in Language, UC San Diego 0526, 9500 Gilman Dr., La Jolla, CA 92093, USA; Swartz Center for Computational Neuroscience, UC San Diego 0559, 9500 Gilman Dr., La Jolla, CA 92093, USA. Electronic address: ywu@cogsci.ucsd.edu.
authors.forename: Ying Choon
authors.initials: YC
authors.lastname: Wu
authors.affiliation: Center for Research in Language, UC San Diego 0526, 9500 Gilman Dr., La Jolla, CA 92093, USA; UC San Diego, Dept. of Cognitive Science 0515, 9500 Gilman Dr., La Jolla, CA 92093, USA.
authors.forename: Seana
authors.initials: S
authors.lastname: Coulson
mesh_terms: D000328:Adult; D032882:Comprehension / Q000502:physiology; D005260:Female; D005868:Gestures; D006801:Humans; D007206:Individuality; D008297:Male; D008570:Memory, Short-Term / Q000502:physiology*; D011597:Psychomotor Performance / Q000502:physiology*; D013028:Space Perception / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*; D055815:Young Adult
keywords: Dual task methodology; Gesture comprehension; Iconic gestures; Multi-modal discourse; Spatial cognition; Working memory