title: Gesture and speech integration: an exploratory study of a man with aphasia.
abstract: BACKGROUND
  In order to comprehend fully a speaker's intention in everyday communication, information is integrated from multiple sources, including gesture and speech. There are no published studies that have explored the impact of aphasia on iconic co-speech gesture and speech integration.
  
  
  OBJECTIVE
  To explore the impact of aphasia on co-speech gesture and speech integration in one participant with aphasia and 20 age-matched control participants.
  
  
  METHODS
  The participant with aphasia and 20 control participants watched video vignettes of people producing 21 verb phrases in three different conditions, verbal only (V), gesture only (G), and verbal gesture combined (VG). Participants were required to select a corresponding picture from one of four alternatives: integration target, a verbal-only match, a gesture-only match, and an unrelated foil. The probability of choosing the integration target in the VG that goes beyond what is expected from the probabilities of choosing the integration target in V and G was referred to as multi-modal gain (MMG).
  
  
  RESULTS
  The participant with aphasia obtained a significantly lower multi-modal gain score than the control participants (p<0.05). Error analysis indicated that in speech and gesture integration tasks, the participant with aphasia relied on gesture in order to decode the message, whereas the control participants relied on speech in order to decode the message. Further analysis of the speech-only and gesture-only tasks indicated that the participant with aphasia had intact gesture comprehension but impaired spoken word comprehension.
  
  
  CONCLUSIONS
  The results confirm findings by Records ( 1994 ) that reported that impaired verbal comprehension leads to a greater reliance on gesture to decode messages. Moreover, multi-modal integration of information from speech and iconic gesture can be impaired in aphasia. The findings highlight the need for further exploration of the impact of aphasia on gesture and speech integration.
authors.affiliation: Language and Communication Science, City University, London, UK. naomi.cocks.1@city.ac.uk
authors.forename: Naomi
authors.initials: N
authors.lastname: Cocks
authors.forename: Laetitia
authors.initials: L
authors.lastname: Sautin
authors.forename: Sotaro
authors.initials: S
authors.lastname: Kita
authors.forename: Gary
authors.initials: G
authors.lastname: Morgan
authors.forename: Sally
authors.initials: S
authors.lastname: Zlotowitz
mesh_terms: D000161:Acoustic Stimulation / Q000379:methods; D000368:Aged; D001039:Aphasia, Broca / Q000523:psychology*; D032882:Comprehension; D005260:Female; D005868:Gestures; D006801:Humans; D008297:Male; D008875:Middle Aged; D010775:Photic Stimulation / Q000379:methods; D013067:Speech Perception; D014741:Video Recording; D014796:Visual Perception