title: Integrating probabilistic models of perception and interactive neural networks: a historical and tutorial review.
abstract: This article seeks to establish a rapprochement between explicitly Bayesian models of contextual effects in perception and neural network models of such effects, particularly the connectionist interactive activation (IA) model of perception. The article is in part an historical review and in part a tutorial, reviewing the probabilistic Bayesian approach to understanding perception and how it may be shaped by context, and also reviewing ideas about how such probabilistic computations may be carried out in neural networks, focusing on the role of context in interactive neural networks, in which both bottom-up and top-down signals affect the interpretation of sensory inputs. It is pointed out that connectionist units that use the logistic or softmax activation functions can exactly compute Bayesian posterior probabilities when the bias terms and connection weights affecting such units are set to the logarithms of appropriate probabilistic quantities. Bayesian concepts such the prior, likelihood, (joint and marginal) posterior, probability matching and maximizing, and calculating vs. sampling from the posterior are all reviewed and linked to neural network computations. Probabilistic and neural network models are explicitly linked to the concept of a probabilistic generative model that describes the relationship between the underlying target of perception (e.g., the word intended by a speaker or other source of sensory stimuli) and the sensory input that reaches the perceiver for use in inferring the underlying target. It is shown how a new version of the IA model called the multinomial interactive activation (MIA) model can sample correctly from the joint posterior of a proposed generative model for perception of letters in words, indicating that interactive processing is fully consistent with principled probabilistic computation. Ways in which these computations might be realized in real neural systems are also considered.
authors: Department of Psychology and Center for Mind, Brain, and Computation, Stanford University Stanford, CA, USA.
authors: James L
authors: JL
authors: McClelland
keywords: context in perception; generative models; interactive activation; neural networks; probabilistic computation
references: Behav Brain Sci. 2000 Jun;23(3):299-325; discussion 325-70
references: 11301575
references: Psychol Rev. 2011 Apr;118(2):247-79
references: 21355661
references: Psychol Rev. 1996 Jul;103(3):518-65
references: 8759046
references: Cogn Psychol. 1991 Jan;23(1):1-44
references: 2001614
references: Bull Math Biol. 1990;52(1-2):99-115; discussion 73-97
references: 2185863
references: J Exp Psychol. 1973 Apr;98(1):49-54
references: 4704213
references: Psychol Rev. 2008 Apr;115(2):357-95
references: 18426294
references: Cogn Sci. 2014 Aug;38(6):1139-89
references: 25098813
references: Science. 2006 Jul 28;313(5786):504-7
references: 16873662
references: IEEE Trans Pattern Anal Mach Intell. 1984 Jun;6(6):721-41
references: 22499653
references: Acta Psychol (Amst). 1979 Sep;43(5):413-26
references: 495176
references: Science. 1974 Jun 14;184(4142):1192-4
references: 4833256
references: Proc Natl Acad Sci U S A. 2010 Sep 14;107(37):16401-6
references: 20805507
references: J Exp Psychol. 1969 Aug;81(2):275-80
references: 5811803
references: J Exp Psychol. 1951 May;41(5):329-35
references: 14861384
references: Br J Psychol. 1990 Aug;81 ( Pt 3):361-80
references: 2224396
references: Psychol Rev. 1982 Jan;89(1):60-94
references: 7058229
references: J Exp Psychol Hum Percept Perform. 1980 Feb;6(1):110-25
references: 6444985
references: Cogn Psychol. 1986 Jan;18(1):1-86
references: 3753912
references: Psychol Rev. 2001 Jan;108(1):113-48
references: 11212625
references: Can J Psychol. 1964 Mar;18:62-71
references: 14129483
references: Proc Natl Acad Sci U S A. 1982 Apr;79(8):2554-8
references: 6953413
references: J Exp Psychol Hum Percept Perform. 1979 Nov;5(4):595-609
references: 528961
references: J Exp Psychol Learn Mem Cogn. 1984 Jan;10(1):104-14
references: 6242730
references: Cogn Psychol. 1989 Jul;21(3):398-421
references: 2758786
references: Cogn Psychol. 1991 Oct;23(4):558-614
references: 1742982
references: J Opt Soc Am A Opt Image Sci Vis. 2003 Jul;20(7):1434-48
references: 12868647
references: Trends Cogn Sci. 2006 Aug;10(8):363-9
references: 16843037
references: Science. 1994 Apr 8;264(5156):231-7
references: 8146653
references: Psychol Rev. 1974 Mar;81(2):99-118
references: 4817613
references: Psychol Rev. 2012 Jul;119(3):573-616
references: 22775499