Past-future information bottleneck in dynamical systems.
Biological systems need to process information in real time and must trade off accuracy of presentation and coding costs. Here we operationalize this trade-off and develop an information-theoretic framework that selectively extracts information of the input past that is predictive about the output future, obtaining a generalized eigenvalue problem. Thereby, we unravel the input history in terms of structural phase transitions corresponding to additional dimensions of a state space. We elucidate the relation to canonical correlation analysis and give a numerical example. Altogether, this work relates information-theoretic optimization to the joint problem of system identification and model reduction.
Creutzig
Felix
F
Berkeley Institute of the Environment, University of California, Berkeley, California 94720, USA. creutzig@nature.berkeley.edu
Globerson
Amir
A
Tishby
Naftali
N
D000465:Algorithms; D005544:Forecasting; D007257:Information Theory; D008954:Models, Biological; D008962:Models, Theoretical