title: Coverage-adjusted entropy estimation.
abstract: Data on 'neural coding' have frequently been analyzed using information-theoretic measures. These formulations involve the fundamental and generally difficult statistical problem of estimating entropy. We review briefly several methods that have been advanced to estimate entropy and highlight a method, the coverage-adjusted entropy estimator (CAE), due to Chao and Shen that appeared recently in the environmental statistics literature. This method begins with the elementary Horvitz-Thompson estimator, developed for sampling from a finite population, and adjusts for the potential new species that have not yet been observed in the sample-these become the new patterns or 'words' in a spike train that have not yet been observed. The adjustment is due to I. J. Good, and is called the Good-Turing coverage estimate. We provide a new empirical regularization derivation of the coverage-adjusted probability estimator, which shrinks the maximum likelihood estimate. We prove that the CAE is consistent and first-order optimal, with rate O(P)(1/log n), in the class of distributions with finite entropy variance and that, within the class of distributions with finite qth moment of the log-likelihood, the Good-Turing coverage estimate and the total probability of unobserved words converge at rate O(P)(1/(log n)(q)). We then provide a simulation study of the estimator with standard distributions and examples from neuronal data, where observations are dependent. The results show that, with a minor modification, the CAE performs much better than the MLE and is better than the best upper bound estimator, due to Paninski, when the number of possible words m is unknown or infinite.
authors.affiliation: Department of Statistics, University of California, Berkeley, CA 94720-3860, USA. vqv@stat.berkeley.edu
authors.forename: Vincent Q
authors.initials: VQ
authors.lastname: Vu
authors.forename: Bin
authors.initials: B
authors.lastname: Yu
authors.forename: Robert E
authors.initials: RE
authors.lastname: Kass
mesh_terms: D000200:Action Potentials / Q000502:physiology*; D001699:Biometry / Q000379:methods*; D019277:Entropy; D006801:Humans; D016013:Likelihood Functions; D008959:Models, Neurological; D015233:Models, Statistical; D009474:Neurons / Q000502:physiology; D014481:United States