title: Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction.
abstract: We prove that maximization of mutual information between the output and the input of a feedforward neural network leads to full redundancy reduction under the following sufficient conditions: (i) the input signal is a (possibly nonlinear) invertible mixture of independent components; (ii) there is no input noise; (iii) the activity of each output neuron is a (possibly) stochastic variable with a probability distribution depending on the stimulus through a deterministic function of the inputs (where both the probability distributions and the functions can be different from neuron to neuron); (iv) optimization of the mutual information is performed over all these deterministic functions. This result extends that obtained by Nadal and Parga (1994) who considered the case of deterministic outputs.
authors.affiliation: Laboratoire de Physique Statistique, Ecole Normale Supérieure, Paris, France.
authors.forename: J P
authors.initials: JP
authors.lastname: Nadal
authors.forename: N
authors.initials: N
authors.lastname: Brunel
authors.forename: N
authors.initials: N
authors.lastname: Parga
mesh_terms: D005246:Feedback / Q000502:physiology; D007257:Information Theory; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D017711:Nonlinear Dynamics; D013269:Stochastic Processes