Explaining Zipf's law via a mental lexicon.
Zipf's law is the major regularity of statistical linguistics that has served as a prototype for rank-frequency relations and scaling laws in natural sciences. Here we show that Zipf's law-together with its applicability for a single text and its generalizations to high and low frequencies including hapax legomena-can be derived from assuming that the words are drawn into the text with random probabilities. Their a priori density relates, via the Bayesian statistics, to the mental lexicon of the author who produced the text.
Laboratoire de Physique Statistique et Systèmes Complexes, ISMANS, 44 ave. Bartholdi, 72000 Le Mans, France and Yerevan Physics Institute, Alikhanian Brothers Street 2, Yerevan 375036, Armenia.
Armen E
AE
Allahverdyan
Laboratoire de Physique Statistique et Systèmes Complexes, ISMANS, 44 ave. Bartholdi, 72000 Le Mans, France and IMMM, UMR CNRS 6283, Université du Maine, 72085 Le Mans, France and Complexity Science Center and Institute of Particle Physics, Hua-Zhong Normal University, Wuhan 430079, China.
Weibing
W
Deng
Laboratoire de Physique Statistique et Systèmes Complexes, ISMANS, 44 ave. Bartholdi, 72000 Le Mans, France and IMMM, UMR CNRS 6283, Université du Maine, 72085 Le Mans, France.
Q A
QA
Wang