title: Word lengths are optimized for efficient communication.
abstract: We demonstrate a substantial improvement on one of the most celebrated empirical laws in the study of language, Zipf's 75-y-old theory that word length is primarily determined by frequency of use. In accord with rational theories of communication, we show across 10 languages that average information content is a much better predictor of word length than frequency. This indicates that human lexicons are efficiently structured for communication by taking into account interword statistical dependencies. Lexical systems result from an optimization of communicative pressures, coding meanings efficiently given the complex statistics of natural language use.
authors.affiliation: Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA. piantado@mit.edu
authors.forename: Steven T
authors.initials: ST
authors.lastname: Piantadosi
authors.forename: Harry
authors.initials: H
authors.lastname: Tily
authors.forename: Edward
authors.initials: E
authors.lastname: Gibson
mesh_terms: D003142:Communication; D006801:Humans; D007802:Language; D015233:Models, Statistical