Mutual information of population codes and distance measures in probability space.
We studied the mutual information between a stimulus and a system consisting of stochastic, statistically independent elements that respond to a stimulus. Using statistical mechanical methods the properties of the mutual information (MI) in the limit of a large system size N are calculated. For continuous valued stimuli, the MI increases logarithmically with N and is related to the log of the Fisher information of the system. For discrete stimuli the MI saturates exponentially with N. We find that the exponent of saturation of the MI is the Chernoff distance between response probabilities that are induced by different stimuli.
Kang
K
K
Racah Institute of Physics and Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel.
Sompolinsky
H
H
D001921:Brain / Q000502:physiology; D003627:Data Interpretation, Statistical; D001330:Electronic Data Processing; D008606:Mental Processes / Q000502:physiology; D008959:Models, Neurological; D009474:Neurons / Q000502:physiology; D011336:Probability; D013269:Stochastic Processes