title: Excess entropy in natural language: Present state and perspectives.
abstract: We review recent progress in understanding the meaning of mutual information in natural language. Let us define words in a text as strings that occur sufficiently often. In a few previous papers, we have shown that a power-law distribution for so defined words (a.k.a. Herdan's law) is obeyed if there is a similar power-law growth of (algorithmic) mutual information between adjacent portions of texts of increasing length. Moreover, the power-law growth of information holds if texts describe a complicated infinite (algorithmically) random object in a highly repetitive way, according to an analogous power-law distribution. The described object may be immutable (like a mathematical or physical constant) or may evolve slowly in time (like cultural heritage). Here, we reflect on the respective mathematical results in a less technical way. We also discuss feasibility of deciding to what extent these results apply to the actual human communication.
authors.affiliation: Institute of Computer Science, Polish Academy of Sciences, Warszawa, Poland. ldebowsk@ipipan.waw.pl
authors.forename: Lukasz
authors.initials: L
authors.lastname: Debowski
mesh_terms: D019277:Entropy; D006801:Humans; D007802:Language; D008037:Linguistics; D008962:Models, Theoretical