title: Past-future information bottleneck in dynamical systems.
abstract: Biological systems need to process information in real time and must trade off accuracy of presentation and coding costs. Here we operationalize this trade-off and develop an information-theoretic framework that selectively extracts information of the input past that is predictive about the output future, obtaining a generalized eigenvalue problem. Thereby, we unravel the input history in terms of structural phase transitions corresponding to additional dimensions of a state space. We elucidate the relation to canonical correlation analysis and give a numerical example. Altogether, this work relates information-theoretic optimization to the joint problem of system identification and model reduction.
authors.affiliation: Berkeley Institute of the Environment, University of California, Berkeley, California 94720, USA. creutzig@nature.berkeley.edu
authors.forename: Felix
authors.initials: F
authors.lastname: Creutzig
authors.forename: Amir
authors.initials: A
authors.lastname: Globerson
authors.forename: Naftali
authors.initials: N
authors.lastname: Tishby
mesh_terms: D000465:Algorithms; D005544:Forecasting; D007257:Information Theory; D008954:Models, Biological; D008962:Models, Theoretical