title: The equivalence of information-theoretic and likelihood-based methods for neural dimensionality reduction.
abstract: Stimulus dimensionality-reduction methods in neuroscience seek to identify a low-dimensional space of stimulus features that affect a neuron's probability of spiking. One popular method, known as maximally informative dimensions (MID), uses an information-theoretic quantity known as "single-spike information" to identify this space. Here we examine MID from a model-based perspective. We show that MID is a maximum-likelihood estimator for the parameters of a linear-nonlinear-Poisson (LNP) model, and that the empirical single-spike information corresponds to the normalized log-likelihood under a Poisson model. This equivalence implies that MID does not necessarily find maximally informative stimulus dimensions when spiking is not well described as Poisson. We provide several examples to illustrate this shortcoming, and derive a lower bound on the information lost when spiking is Bernoulli in discrete time bins. To overcome this limitation, we introduce model-based dimensionality reduction methods for neurons with non-Poisson firing statistics, and show that they can be framed equivalently in likelihood-based or information-theoretic terms. Finally, we show how to overcome practical limitations on the number of stimulus dimensions that MID can estimate by constraining the form of the non-parametric nonlinearity in an LNP model. We illustrate these methods with simulations and data from primate visual cortex.
authors.affiliation: Gatsby Computational Neuroscience Unit, University College London, London, UK; Centre for Mathematics and Physics in the Life Sciences and Experimental Biology, University College London, London, UK.
authors.forename: Ross S
authors.initials: RS
authors.lastname: Williamson
authors.affiliation: Gatsby Computational Neuroscience Unit, University College London, London, UK.
authors.forename: Maneesh
authors.initials: M
authors.lastname: Sahani
authors.affiliation: Princeton Neuroscience Institute, Department of Psychology, Princeton University, Princeton, New Jersey, USA.
authors.forename: Jonathan W
authors.initials: JW
authors.lastname: Pillow
mesh_terms: D000200:Action Potentials / Q000502:physiology*; D000818:Animals; D003198:Computer Simulation; D006801:Humans; D007257:Information Theory; D016013:Likelihood Functions; D008959:Models, Neurological; D015233:Models, Statistical; D009474:Neurons / Q000502:physiology*; D009435:Synaptic Transmission / Q000502:physiology*
references.citation: Adv Neural Inf Process Syst. 2012 Dec;25:3113-3121
references.pmid: 26273181
references.citation: Proc Natl Acad Sci U S A. 2009 Dec 22;106(51):21894-9
references.pmid: 19918079
references.citation: J Neurosci. 2013 Oct 16;33(42):16715-28
references.pmid: 24133273
references.citation: J Neurophysiol. 2012 May;107(10):2594-603
references.pmid: 22323634
references.citation: Ann Biomed Eng. 1988;16(2):201-14
references.pmid: 3382067
references.citation: J Neurophysiol. 2005 Feb;93(2):1074-89
references.pmid: 15356183
references.citation: Nat Neurosci. 2006 Nov;9(11):1421-31
references.pmid: 17041595
references.citation: Neural Comput. 2013 Jul;25(7):1661-92
references.pmid: 23607557
references.citation: Neural Comput. 2004 Feb;16(2):223-50
references.pmid: 15006095
references.citation: J Neurosci. 2008 Jan 9;28(2):446-55
references.pmid: 18184787
references.citation: J Vis. 2006 Jul 17;6(4):484-507
references.pmid: 16889482
references.citation: Neural Comput. 1991 Summer;3(2):246-257
references.pmid: 31167308
references.citation: Network. 2003 Aug;14(3):437-64
references.pmid: 12938766
references.citation: J Neurosci. 2000 Mar 1;20(5):1964-74
references.pmid: 10684897
references.citation: J Neurosci. 2004 Sep 29;24(39):8551-61
references.pmid: 15456829
references.citation: Neural Comput. 2003 Aug;15(8):1789-807
references.pmid: 14511513
references.citation: Curr Opin Neurobiol. 2011 Oct;21(5):761-7
references.pmid: 21704508
references.citation: PLoS Comput Biol. 2011 Oct;7(10):e1002219
references.pmid: 22046110
references.citation: Proc Natl Acad Sci U S A. 2007 Jan 9;104(2):501-6
references.pmid: 17197415
references.citation: Neural Comput. 2003 Aug;15(8):1715-49
references.pmid: 14511510
references.citation: Neural Comput. 2000 Jul;12(7):1531-52
references.pmid: 10935917
references.citation: Network. 2011;22(1-4):45-73
references.pmid: 21780916
references.citation: Network. 2004 Nov;15(4):243-62
references.pmid: 15600233
references.citation: J Vis. 2008 Oct 13;8(13):2.1-16
references.pmid: 19146332
references.citation: Neuron. 2008 Jun 26;58(6):956-66
references.pmid: 18579084
references.citation: J Vis. 2006 Apr 28;6(4):414-28
references.pmid: 16889478
references.citation: Neural Comput. 2013 Mar;25(3):725-58
references.pmid: 23272920
references.citation: PLoS Comput Biol. 2013;9(7):e1003143
references.pmid: 23874185
references.citation: PLoS Comput Biol. 2013;9(11):e1003356
references.pmid: 24278006
references.citation: J Neurosci Methods. 2001 Jan 30;105(1):25-37
references.pmid: 11166363
references.citation: PLoS Comput Biol. 2011 Oct;7(10):e1002249
references.pmid: 22046122
references.citation: PLoS Comput Biol. 2011 Mar;7(3):e1001111
references.pmid: 21455284
references.citation: Nature. 2006 Apr 20;440(7087):1007-12
references.pmid: 16625187
references.citation: Nature. 2008 Aug 21;454(7207):995-9
references.pmid: 18650810
references.citation: Neural Comput. 2002 Aug;14(8):1771-800
references.pmid: 12180402
references.citation: Biol Cybern. 1988;59(3):189-200
references.pmid: 3179344
references.citation: Ann Biomed Eng. 1991;19(4):383-99
references.pmid: 1741523
references.citation: Network. 2010;21(1-2):35-90
references.pmid: 20735338
references.citation: Neuron. 2009 May 14;62(3):426-40
references.pmid: 19447097
references.citation: Network. 2010;21(3-4):142-68
references.pmid: 21138363
references.citation: Cereb Cortex. 2014 Dec;24(12):3232-45
references.pmid: 23863686
references.citation: Network. 2001 May;12(2):199-213
references.pmid: 11405422
references.citation: Neural Netw. 2010 Jan;23(1):44-59
references.pmid: 19631506
references.citation: Neural Comput. 2001 Aug;13(8):1713-20
references.pmid: 11506667
references.citation: PLoS One. 2013 Nov 08;8(11):e71959
references.pmid: 24250780
references.citation: Network. 2009;20(2):49-68
references.pmid: 19568981
references.citation: Neuron. 2005 Jun 16;46(6):945-56
references.pmid: 15953422