title: Language as a multimodal phenomenon: implications for language learning, processing and evolution.
abstract: Our understanding of the cognitive and neural underpinnings of language has traditionally been firmly based on spoken Indo-European languages and on language studied as speech or text. However, in face-to-face communication, language is multimodal: speech signals are invariably accompanied by visual information on the face and in manual gestures, and sign languages deploy multiple channels (hands, face and body) in utterance construction. Moreover, the narrow focus on spoken Indo-European languages has entrenched the assumption that language is comprised wholly by an arbitrary system of symbols and rules. However, iconicity (i.e. resemblance between aspects of communicative form and meaning) is also present: speakers use iconic gestures when they speak; many non-Indo-European spoken languages exhibit a substantial amount of iconicity in word forms and, finally, iconicity is the norm, rather than the exception in sign languages. This introduction provides the motivation for taking a multimodal approach to the study of language learning, processing and evolution, and discusses the broad implications of shifting our current dominant approaches and assumptions to encompass multimodal expression in both signed and spoken languages.
authors.affiliation: Cognitive, Perceptual & Brain Sciences Department, 26 Bedford Way, London WC1H 0AP Deafness, Cognition & Language Research Centre, 49 Gordon Square, London WC1H 0PD g.vigliocco@ucl.ac.uk.
authors.forename: Gabriella
authors.initials: G
authors.lastname: Vigliocco
authors.affiliation: Cognitive, Perceptual & Brain Sciences Department, 26 Bedford Way, London WC1H 0AP Deafness, Cognition & Language Research Centre, 49 Gordon Square, London WC1H 0PD.
authors.forename: Pamela
authors.initials: P
authors.lastname: Perniss
authors.affiliation: Cognitive, Perceptual & Brain Sciences Department, 26 Bedford Way, London WC1H 0AP.
authors.forename: David
authors.initials: D
authors.lastname: Vinson
mesh_terms: D005075:Biological Evolution; D006801:Humans; D007802:Language; D007858:Learning / Q000502:physiology*; D008037:Linguistics / Q000639:trends*; D009633:Nonverbal Communication / Q000502:physiology*; D012660:Semantics
keywords: iconicity; language development; language evolution; language processing; multimodality of language
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130294
references.pmid: 25092662
references.citation: Nature. 1976 Dec 23-30;264(5588):746-8
references.pmid: 1012311
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130303
references.pmid: 25092671
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130296
references.pmid: 25092664
references.citation: J Exp Psychol Learn Mem Cogn. 2010 Nov;36(6):1573-81
references.pmid: 20919784
references.citation: Behav Res Methods. 2008 Nov;40(4):1079-87
references.pmid: 19001399
references.citation: Annu Rev Psychol. 2011;62:621-47
references.pmid: 20809790
references.citation: Lang Cogn Process. 2013 Mar 1;28(3):261-271
references.pmid: 23543899
references.citation: Brain Lang. 2005 Apr;93(1):10-9
references.pmid: 15766764
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130302
references.pmid: 25092670
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130300
references.pmid: 25092668
references.citation: Dev Sci. 2006 May;9(3):316-22
references.pmid: 16669803
references.citation: Front Psychol. 2010 Dec 31;1:227
references.pmid: 21833282
references.citation: Behav Brain Sci. 1999 Feb;22(1):1-38; discussion 38-75
references.pmid: 11301520
references.citation: Cognition. 2010 Jan;114(1):19-28
references.pmid: 19828141
references.citation: J Child Lang. 2005 Aug;32(3):481-505
references.pmid: 16220632
references.citation: Science. 2009 Feb 13;323(5916):951-3
references.pmid: 19213922
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130297
references.pmid: 25092665
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130298
references.pmid: 25092666
references.citation: Cognition. 2008 Oct;109(1):54-65
references.pmid: 18835600
references.citation: J Exp Psychol Learn Mem Cogn. 2009 Mar;35(2):550-7
references.pmid: 19271866
references.citation: J Speech Hear Disord. 1985 Aug;50(3):299-301
references.pmid: 4021459
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130301
references.pmid: 25092669
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130293
references.pmid: 25092661
references.citation: J Exp Psychol Learn Mem Cogn. 2010 Jul;36(4):1017-27
references.pmid: 20565217
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130299
references.pmid: 25092667
references.citation: Cognition. 1986 Jun;23(1):43-71
references.pmid: 3742990
references.citation: J Speech Hear Disord. 1984 Aug;49(3):287-92
references.pmid: 6748624
references.citation: Psychol Sci. 2010 Feb;21(2):260-7
references.pmid: 20424055
references.citation: Language (Baltim). 2005 Jun;81(2):301-344
references.pmid: 22223926
references.citation: Semiotica. 2009 Apr;2009(174):241-275
references.pmid: 20445832
references.citation: Trends Cogn Sci. 2003 Feb;7(2):84-91
references.pmid: 12584027
references.citation: Psychophysiology. 2005 Nov;42(6):654-67
references.pmid: 16364061
references.citation: Psychol Sci. 2009 Mar;20(3):267-72
references.pmid: 19222810
references.citation: PLoS One. 2014 May 19;9(5):e97905
references.pmid: 24840874
references.citation: Cognition. 2009 Jul;112(1):181-6
references.pmid: 19447384
references.citation: Trends Cogn Sci. 2014 May;18(5):229-34
references.pmid: 24630873
references.citation: Philos Trans R Soc Lond B Biol Sci. 2014 Sep 19;369(1651):20130295
references.pmid: 25092663
references.citation: Trends Cogn Sci. 2013 May;17(5):202-9
references.pmid: 23608361
references.citation: Proc Biol Sci. 2005 Sep 7;272(1574):1859-63
references.pmid: 16096100
references.citation: Psychol Sci. 2012 Dec;23(12):1443-8
references.pmid: 23150275