title: Signatures of infinity: Nonergodicity and resource scaling in prediction, complexity, and learning.
abstract: We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multiarm Bandit problem. We contrast our analysis with computation-theoretic and statistical inference approaches to understanding their complexity. The result is an alternative view of the relationship between predictability, complexity, and learning that highlights the distinct ways in which informational and correlational divergences arise in complex ergodic and nonergodic processes. We draw out consequences for the resource divergences that delineate the structural hierarchy of ergodic processes and for processes that are themselves hierarchical.
authors.affiliation: Complexity Sciences Center and Department of Physics, University of California at Davis, One Shields Avenue, Davis, California 95616, USA.
authors.forename: James P
authors.initials: JP
authors.lastname: Crutchfield
authors.affiliation: Redwood Center for Theoretical Neuroscience and Department of Physics, University of California at Berkeley, Berkeley, California 94720-5800, USA.
authors.forename: Sarah
authors.initials: S
authors.lastname: Marzen
mesh_terms: D000069550:Machine Learning; D013997:Time Factors