title: Learning multisensory integration and coordinate transformation via density estimation.
abstract: Sensory processing in the brain includes three key operations: multisensory integration-the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations-the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned-but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.
authors: Department of Physiology and the Center for Integrative Neuroscience, University of California San Francisco, San Francisco, California, USA. makin@phy.ucsf.edu
authors: Joseph G
authors: JG
authors: Makin
authors: Matthew R
authors: MR
authors: Fellows
authors: Philip N
authors: PN
authors: Sabes
mesh_terms: D000465:Algorithms; D000818:Animals; D001921:Brain / Q000502:physiology; D003463:Cues; D006801:Humans; D007858:Learning; D008959:Models, Neurological; D016011:Normal Distribution; D010775:Photic Stimulation; D016012:Poisson Distribution; D011336:Probability; D012677:Sensation; D014796:Visual Perception
references: Neural Comput. 2006 Jul;18(7):1527-54
references: 16764513
references: J Neurosci. 2003 Aug 6;23(18):6982-92
references: 12904459
references: J Neurosci. 1989 Sep;9(9):3306-13
references: 2795164
references: Eur J Neurosci. 1998 Oct;10(10):3171-93
references: 9786211
references: Nat Neurosci. 2006 Apr;9(4):578-85
references: 16547513
references: Proc Natl Acad Sci U S A. 2010 Apr 27;107(17):7951-6
references: 20375282
references: Trends Neurosci. 1990 Jun;13(6):227-33
references: 1694329
references: Cereb Cortex. 1996 Mar-Apr;6(2):102-19
references: 8670643
references: Nat Neurosci. 2009 Aug;12(8):1056-61
references: 19597495
references: Vision Res. 1997 Dec;37(23):3311-25
references: 9425546
references: Nature. 1997 Oct 23;389(6653):845-8
references: 9349815
references: Psychol Rev. 1954 May;61(3):183-93
references: 13167245
references: Neuropsychologia. 2006;44(13):2594-606
references: 16300804
references: Proc Natl Acad Sci U S A. 1993 May 15;90(10):4650-4
references: 8506312
references: Proc Natl Acad Sci U S A. 1999 Aug 31;96(18):10418-21
references: 10468623
references: Neuron. 2012 Jul 26;75(2):342-51
references: 22841318
references: Neuron. 2006 Jul 6;51(1):125-34
references: 16815337
references: Nat Neurosci. 2005 Jul;8(7):941-9
references: 15951810
references: PLoS One. 2007 Sep 26;2(9):e943
references: 17895984
references: Proc Natl Acad Sci U S A. 1989 Dec;86(23):9574-8
references: 2556718
references: J Comp Neurol. 2000 Dec 4;428(1):112-37
references: 11058227
references: Eur J Neurosci. 2001 Apr;13(8):1572-88
references: 11328351
references: Trends Neurosci. 2004 Dec;27(12):712-9
references: 15541511
references: Nat Neurosci. 2005 Apr;8(4):490-7
references: 15793578
references: J Neurosci. 2006 May 24;26(21):5604-15
references: 16723517
references: J Neurophysiol. 1998 Jan;79(1):126-36
references: 9425183
references: Cogn Sci. 2012 Mar;36(2):305-32
references: 22141921
references: Neural Comput. 2005 Oct;17(10):2215-39
references: 16105223
references: Nature. 1992 Jan 9;355(6356):161-3
references: 1729650
references: J Neurosci. 1992 Apr;12(4):1435-53
references: 1556602
references: Nat Rev Neurosci. 2002 Sep;3(9):741-7
references: 12209122
references: J Neurosci. 1996 Nov 1;16(21):7085-96
references: 8824344
references: J Comp Physiol Psychol. 1963 Oct;56:872-6
references: 14050177
references: Network. 2001 Aug;12(3):241-53
references: 11563528
references: Annu Rev Neurosci. 1997;20:25-42
references: 9056706
references: J Neurophysiol. 1997 Feb;77(2):1034-8
references: 9065868
references: Trends Cogn Sci. 2004 Apr;8(4):162-9
references: 15050512
references: Nature. 2004 Jan 15;427(6971):244-7
references: 14724638
references: J Neurosci. 2011 May 4;31(18):6661-73
references: 21543595
references: Nat Neurosci. 2002 Apr;5(4):356-63
references: 11896400
references: J Neurosci. 2011 Jul 6;31(27):10050-9
references: 21734297
references: Science. 2006 Jul 28;313(5786):504-7
references: 16873662
references: Curr Opin Neurobiol. 1998 Apr;8(2):195-201
references: 9635202
references: Curr Biol. 2004 Feb 3;14(3):257-62
references: 14761661
references: Nat Neurosci. 2001 Aug;4(8):826-31
references: 11477429
references: Nat Neurosci. 2011 Nov 20;15(1):146-54
references: 22101645
references: Neural Comput. 2002 Aug;14(8):1771-800
references: 12180402
references: J Neurophysiol. 2007 Nov;98(5):2827-41
references: 17728389
references: Prog Brain Res. 2011;191:195-209
references: 21741553
references: J Cogn Neurosci. 2000 Jul;12(4):601-14
references: 10936913
references: J Neurosci. 1995 Oct;15(10):6461-74
references: 7472409
references: Nat Neurosci. 2006 Nov;9(11):1432-8
references: 17057707
references: Science. 1963 Oct 25;142(3591):455-62
references: 14064442
references: PLoS Comput Biol. 2009 Apr;5(4):e1000336
references: 19343216
references: Results Probl Cell Differ. 2002;39:139-56
references: 12353467
references: Cereb Cortex. 2003 Oct;13(10):1009-22
references: 12967918
references: J Neurophysiol. 1999 Mar;81(3):1355-64
references: 10085361
references: Neurosci Biobehav Rev. 2005 May;29(3):431-44
references: 15820548
references: Nature. 2002 Jan 24;415(6870):429-33
references: 11807554
references: Phys Rev Lett. 2011 Nov 25;107(22):220601
references: 22182019