title: Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel.
abstract: Successful speech perception requires that listeners map the acoustic signal to linguistic categories. These mappings are not only probabilistic, but change depending on the situation. For example, one talker's /p/ might be physically indistinguishable from another talker's /b/ (cf. lack of invariance). We characterize the computational problem posed by such a subjectively nonstationary world and propose that the speech perception system overcomes this challenge by (a) recognizing previously encountered situations, (b) generalizing to other situations based on previous similar experience, and (c) adapting to novel situations. We formalize this proposal in the ideal adapter framework: (a) to (c) can be understood as inference under uncertainty about the appropriate generative model for the current talker, thereby facilitating robust speech perception despite the lack of invariance. We focus on 2 critical aspects of the ideal adapter. First, in situations that clearly deviate from previous experience, listeners need to adapt. We develop a distributional (belief-updating) learning model of incremental adaptation. The model provides a good fit against known and novel phonetic adaptation data, including perceptual recalibration and selective adaptation. Second, robust speech recognition requires that listeners learn to represent the structured component of cross-situation variability in the speech signal. We discuss how these 2 aspects of the ideal adapter provide a unifying explanation for adaptation, talker-specificity, and generalization across talkers and groups of talkers (e.g., accents and dialects). The ideal adapter provides a guiding framework for future investigations into speech perception and adaptation, and more broadly language comprehension.
authors.affiliation: Department of Brain and Cognitive Sciences, University of Rochester.
authors.forename: Dave F
authors.initials: DF
authors.lastname: Kleinschmidt
authors.affiliation: Departments of Brain and Cognitive Sciences, Computer Science, and Linguistics, University of Rochester.
authors.forename: T Florian
authors.initials: TF
authors.lastname: Jaeger
mesh_terms: D000223:Adaptation, Psychological / Q000502:physiology*; D005793:Generalization, Psychological / Q000502:physiology*; D006801:Humans; D021641:Recognition, Psychology / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*
references.citation: Dev Sci. 2009 Apr;12(3):369-78
references.pmid: 19371359
references.citation: Cogn Psychol. 2009 Mar;58(2):250-71
references.pmid: 18922516
references.citation: J Mem Lang. 2014 Feb 1;71(1):145-163
references.pmid: 24511179
references.citation: Cognition. 2016 Dec;157:156-173
references.pmid: 27639552
references.citation: Behav Brain Sci. 2013 Jun;36(3):211-2
references.pmid: 23663410
references.citation: J Acoust Soc Am. 2001 Mar;109(3):1181-96
references.pmid: 11303932
references.citation: Psychol Rev. 1994 Oct;101(4):676-703
references.pmid: 7984711
references.citation: Cognition. 2002 Jan;82(3):B101-11
references.pmid: 11747867
references.citation: Percept Psychophys. 1984 Jan;35(1):49-64
references.pmid: 6709474
references.citation: Proc Natl Acad Sci U S A. 2007 Aug 14;104(33):13273-8
references.pmid: 17664424
references.citation: J Acoust Soc Am. 1995 May;97(5 Pt 1):3099-111
references.pmid: 7759650
references.citation: Nature. 2006 Feb 23;439(7079):936-42
references.pmid: 16495990
references.citation: Behav Brain Sci. 2000 Jun;23(3):299-325; discussion 325-70
references.pmid: 11301575
references.citation: J Exp Psychol Hum Percept Perform. 2011 Dec;37(6):1939-56
references.pmid: 22004192
references.citation: Trends Cogn Sci. 2006 Jul;10(7):294-300
references.pmid: 16793323
references.citation: J Exp Psychol Anim Behav Process. 2001 Oct;27(4):354-72
references.pmid: 11676086
references.citation: Psychon Bull Rev. 2010 Aug;17(4):443-64
references.pmid: 20702863
references.citation: Cognition. 2008 Sep;108(3):710-8
references.pmid: 18653175
references.citation: Front Psychol. 2012 Jul 20;3:228
references.pmid: 22833727
references.citation: Nat Neurosci. 2011 May;14(5):642-8
references.pmid: 21460833
references.citation: Cognition. 1994 Sep;52(3):163-87
references.pmid: 7956004
references.citation: J Acoust Soc Am. 2009 May;125(5):3306-16
references.pmid: 19425672
references.citation: J Exp Psychol Gen. 2005 May;134(2):222-41
references.pmid: 15869347
references.citation: Neuropsychologia. 2007 Feb 1;45(3):572-7
references.pmid: 16530233
references.citation: J Acoust Soc Am. 2011 Nov;130(5):2631-4
references.pmid: 22087888
references.citation: J Exp Psychol Learn Mem Cogn. 2006 Mar;32(2):387-98
references.pmid: 16569154
references.citation: Lang Speech. 2013 Sep;56(Pt 3):309-28
references.pmid: 24416959
references.citation: Percept Psychophys. 1998 Apr;60(3):355-76
references.pmid: 9599989
references.citation: Neuron. 2000 Jun;26(3):695-702
references.pmid: 10896164
references.citation: Science. 1995 Jun 16;268(5217):1632-4
references.pmid: 7777863
references.citation: J Neurophysiol. 2007 May;97(5):3155-64
references.pmid: 17344377
references.citation: Nature. 1976 Dec 23-30;264(5588):746-8
references.pmid: 1012311
references.citation: Cognition. 2008 Mar;106(3):1126-77
references.pmid: 17662975
references.citation: Cognition. 2008 Nov;109(2):193-210
references.pmid: 18930455
references.citation: Cogn Psychol. 2003 Sep;47(2):204-38
references.pmid: 12948518
references.citation: Psychol Sci. 2008 Apr;19(4):332-8
references.pmid: 18399885
references.citation: Front Psychol. 2013 Apr 26;4:226
references.pmid: 23637689
references.citation: Cogn Psychol. 2008 May;56(3):165-209
references.pmid: 17662707
references.citation: Psychol Rev. 1994 Oct;101(4):653-75
references.pmid: 7984710
references.citation: Psychol Rev. 2004 Apr;111(2):395-429
references.pmid: 15065915
references.citation: J Acoust Soc Am. 2004 Dec;116(6):3647-58
references.pmid: 15658715
references.citation: Psychol Sci. 2011 Jun;22(6):829-34
references.pmid: 21586764
references.citation: Behav Neurosci. 2011 Aug;125(4):613-25
references.pmid: 21517145
references.citation: Cogn Affect Behav Neurosci. 2002 Dec;2(4):283-99
references.pmid: 12641174
references.citation: J Acoust Soc Am. 1983 Jun;73(6):2124-33
references.pmid: 6875098
references.citation: Cognition. 2011 Apr;119(1):131-6
references.pmid: 21144500
references.citation: Psychol Rev. 2013 Oct;120(4):751-78
references.pmid: 24219848
references.citation: Behav Brain Sci. 2013 Aug;36(4):359-60
references.pmid: 23789872
references.citation: Psychon Bull Rev. 2006 Dec;13(6):958-65
references.pmid: 17484419
references.citation: PLoS One. 2007 Sep 26;2(9):e943
references.pmid: 17895984
references.citation: Vision Res. 2003 Nov;43(24):2539-58
references.pmid: 13129541
references.citation: Cogn Psychol. 1986 Jan;18(1):1-86
references.pmid: 3753912
references.citation: J Exp Psychol Learn Mem Cogn. 2003 Jul;29(4):539-53
references.pmid: 12924857
references.citation: Dev Sci. 2011 Mar;14(2):372-84
references.pmid: 21479106
references.citation: Cogn Sci. 2013 Mar;37(2):344-77
references.pmid: 23137418
references.citation: Proc Natl Acad Sci U S A. 2005 Jan 25;102(4):1181-6
references.pmid: 15647358
references.citation: Trends Cogn Sci. 2010 Aug;14(8):357-64
references.pmid: 20576465
references.citation: Cognition. 2012 Jul;124(1):66-71
references.pmid: 22498776
references.citation: Nature. 2001 Aug 23;412(6849):787-92
references.pmid: 11518957
references.citation: Psychol Rev. 2011 Apr;118(2):219-46
references.pmid: 21417542
references.citation: Behav Brain Sci. 2013 Aug;36(4):329-47
references.pmid: 23789620
references.citation: J Exp Psychol Hum Percept Perform. 2016 Nov;42(11):1793-1805
references.pmid: 27399829
references.citation: J Exp Psychol Learn Mem Cogn. 2010 May;36(3):821-8
references.pmid: 20438279
references.citation: J Exp Psychol Learn Mem Cogn. 2006 Mar;32(2):425-36
references.pmid: 16569157
references.citation: J Acoust Soc Am. 1994 Jun;95(6):3658-61
references.pmid: 8046153
references.citation: J Acoust Soc Am. 2013 Mar;133(3):EL174-80
references.pmid: 23464125
references.citation: Psychol Rev. 1998 Apr;105(2):251-79
references.pmid: 9577239
references.citation: J Cogn Neurosci. 2012 Apr;24(4):878-87
references.pmid: 21812565
references.citation: Psychol Rev. 1967 Nov;74(6):431-61
references.pmid: 4170865
references.citation: Nat Neurosci. 1999 Jan;2(1):79-87
references.pmid: 10195184
references.citation: Lang Speech. 2007;50(Pt 1):125-42
references.pmid: 17518106
references.citation: Philos Trans R Soc Lond B Biol Sci. 2005 Apr 29;360(1456):815-36
references.pmid: 15937014
references.citation: Psychol Rev. 2009 Oct;116(4):752-782
references.pmid: 19839683
references.citation: Psychol Rev. 2008 Apr;115(2):357-95
references.pmid: 18426294
references.citation: J Exp Psychol Hum Percept Perform. 2007 Apr;33(2):391-409
references.pmid: 17469975
references.citation: Behav Res Methods Instrum Comput. 2004 Aug;36(3):481-7
references.pmid: 15641436
references.citation: J Exp Psychol Learn Mem Cogn. 1996 Sep;22(5):1166-83
references.pmid: 8926483
references.citation: Percept Psychophys. 1985 Oct;38(4):299-310
references.pmid: 3831907
references.citation: Cognition. 2011 Mar;118(3):306-38
references.pmid: 21186021
references.citation: PLoS One. 2011;6(5):e19812
references.pmid: 21637344
references.citation: Nat Neurosci. 2005 Jan;8(1):93-8
references.pmid: 15608636
references.citation: Cognition. 2008 Feb;106(2):707-29
references.pmid: 17532315
references.citation: Psychol Sci. 2003 Nov;14(6):592-7
references.pmid: 14629691
references.citation: Wiley Interdiscip Rev Cogn Sci. 2011 Sep;2(5):580-593
references.pmid: 26302308
references.citation: Annu Rev Psychol. 2004;55:271-304
references.pmid: 14744217
references.citation: Science. 2012 May 25;336(6084):998
references.pmid: 22628647
references.citation: Front Psychol. 2012 Feb 01;3:10
references.pmid: 22347198
references.citation: Lang Speech. 2003;46(Pt 2-3):115-54
references.pmid: 14748442
references.citation: PLoS One. 2013 Oct 30;8(10):e77661
references.pmid: 24204909
references.citation: Psychol Rev. 2006 Apr;113(2):234-72
references.pmid: 16637761
references.citation: Biol Cybern. 2012 Dec;106(11-12):657-68
references.pmid: 22711216
references.citation: Science. 1998 Mar 6;279(5356):1531-3
references.pmid: 9488651
references.citation: J Exp Psychol Learn Mem Cogn. 1993 Mar;19(2):309-28
references.pmid: 8454963
references.citation: J Exp Psychol Anim Behav Process. 1983 Jul;9(3):248-65
references.pmid: 6886630
references.citation: Neuron. 2012 Apr 12;74(1):30-9
references.pmid: 22500627
references.citation: Percept Psychophys. 2008 May;70(4):604-18
references.pmid: 18556922
references.citation: Cogn Sci. 2010 Apr;34(3):434-464
references.pmid: 21339861
references.citation: J Acoust Soc Am. 1966 Dec;40(6):1441-9
references.pmid: 5975580
references.citation: Percept Psychophys. 2005 Feb;67(2):224-38
references.pmid: 15971687
references.citation: J Exp Psychol Anim Behav Process. 2009 Jul;35(3):419-26
references.pmid: 19594286
references.citation: Psychol Sci. 2011 Nov;22(11):1359-66
references.pmid: 22006061
references.citation: J Acoust Soc Am. 1989 May;85(5):2081-7
references.pmid: 2659637
references.citation: Lang Learn. 2016 Dec;66(4):900-944
references.pmid: 28348442
references.citation: J Exp Psychol Learn Mem Cogn. 2000 Nov;26(6):1355-67
references.pmid: 11185769
references.citation: Trends Cogn Sci. 2007 Oct;11(10):428-34
references.pmid: 17921042
references.citation: Curr Opin Neurobiol. 2012 Dec;22(6):956-62
references.pmid: 22695048
references.citation: Psychol Sci. 2001 Jul;12(4):348-51
references.pmid: 11476105
references.citation: Psychol Rev. 1978 May;85(3):172-91
references.pmid: 663005
references.citation: Cogn Psychol. 1986 Oct;18(4):452-99
references.pmid: 3769426
references.citation: J Acoust Soc Am. 1997 Jun;101(6):3241-54
references.pmid: 9193041
references.citation: Front Psychol. 2013 Apr 26;4:230
references.pmid: 23637690
references.citation: Cogn Psychol. 2005 Sep;51(2):141-78
references.pmid: 16095588
references.citation: Cogn Sci. 2006 Nov 12;30(6):1113-26
references.pmid: 21702849
references.citation: Psychol Rev. 2010 Oct;117(4):1144-67
references.pmid: 21038975
references.citation: Cognition. 2002 Dec;86(2):B33-42
references.pmid: 12435537
references.citation: Trends Cogn Sci. 2002 Aug 1;6(8):345-350
references.pmid: 12140085
references.citation: Curr Biol. 2012 Oct 9;22(19):1813-7
references.pmid: 22921366
references.citation: Top Cogn Sci. 2013 Jan;5(1):132-72
references.pmid: 23335577
references.citation: Cogn Sci. 2008 Apr 5;32(3):543-62
references.pmid: 21635345
references.citation: Cognition. 2008 Feb;106(2):633-64
references.pmid: 17507006
references.citation: Cognition. 2008 Sep;108(3):804-9
references.pmid: 18582855
references.citation: Cognition. 2013 Apr;127(1):57-83
references.pmid: 23354056
references.citation: J Exp Psychol Hum Percept Perform. 2002 Dec;28(6):1447-69
references.pmid: 12542137
references.citation: Psychon Bull Rev. 2006 Apr;13(2):262-8
references.pmid: 16892992
references.citation: Science. 1996 Dec 13;274(5294):1926-8
references.pmid: 8943209
references.citation: J Acoust Soc Am. 1977 Sep;62(3):738-50
references.pmid: 903514
references.citation: J Acoust Soc Am. 2004 Jun;115(6):3171-83
references.pmid: 15237841
references.citation: J Acoust Soc Am. 2006 Apr;119(4):1950-3
references.pmid: 16642808
references.citation: Philos Trans R Soc Lond B Biol Sci. 2013 Dec 09;369(1634):20120394
references.pmid: 24324238
references.citation: Behav Brain Sci. 2013 Jun;36(3):181-204
references.pmid: 23663408
references.citation: Behav Brain Sci. 2001 Aug;24(4):629-40; discussion 652-791
references.pmid: 12048947
references.citation: J Exp Psychol Hum Percept Perform. 2014 Apr;40(2):539-55
references.pmid: 24059846
references.citation: J Exp Psychol Hum Percept Perform. 1984 Aug;10(4):512-25
references.pmid: 6235316
references.citation: Nat Neurosci. 2006 Nov;9(11):1432-8
references.pmid: 17057707
references.citation: Percept Psychophys. 1988 Nov;44(5):395-408
references.pmid: 3226889
references.citation: J Mem Lang. 2008 Nov;59(4):434-446
references.pmid: 19884961
references.citation: Top Cogn Sci. 2013 Jan;5(1):173-84
references.pmid: 23335578
references.citation: Front Syst Neurosci. 2014 Jan 03;7:126
references.pmid: 24427119
references.citation: Proc Natl Acad Sci U S A. 2009 Dec 15;106(50):21086-90
references.pmid: 19965371
references.citation: Percept Psychophys. 2001 Oct;63(7):1109-39
references.pmid: 11766939
references.citation: J Acoust Soc Am. 2003 Jan;113(1):544-52
references.pmid: 12558290
references.citation: Percept Psychophys. 1974;15(2):285-290
references.pmid: 23226881
references.citation: Percept Psychophys. 1981 Oct;30(4):309-14
references.pmid: 7322807
references.citation: Cognition. 2013 Sep;128(3):302-19
references.pmid: 23747651
references.citation: Nat Neurosci. 2007 Jun;10(6):779-86
references.pmid: 17496891
references.citation: Psychol Sci. 2009 May;20(5):578-85
references.pmid: 19389131
references.citation: Percept Psychophys. 2007 Jul;69(5):744-56
references.pmid: 17929697
references.citation: J Exp Psychol Gen. 2004 Sep;133(3):450-67
references.pmid: 15355149
references.citation: J Acoust Soc Am. 2000 Sep;108(3 Pt 1):1252-63
references.pmid: 11008825
references.citation: Trends Cogn Sci. 2000 May;4(5):178-186
references.pmid: 10782103
references.citation: Nature. 2002 Jan 24;415(6870):429-33
references.pmid: 11807554
references.citation: J Exp Psychol Hum Percept Perform. 1997 Jun;23(3):651-66
references.pmid: 9180039
references.citation: Cognition. 2013 Jun;127(3):427-38
references.pmid: 23562941