To access the full text documents, please follow this link: http://hdl.handle.net/2445/69418

Modeling the structure of written text
Serrano Moral, Ma. Ángeles (María Ángeles); Flammini, Alessandro; Menczer, Filippo
Universitat de Barcelona
Written text is one of the fundamental manifestations of human language, and the study of its universal regularities can give clues about how our brains process information and how we, as a society, organize and share it. Among these regularities, only Zipf's law has been explored in depth. Other basic properties, such as the existence of bursts of rare words in specific documents, have only been studied independently of each other and mainly by descriptive models. As a consequence, there is a lack of understanding of linguistic processes as complex emergent phenomena. Beyond Zipf's law for word frequencies, here we focus on burstiness, Heaps' law describing the sublinear growth of vocabulary size with the length of a document, and the topicality of document collections, which encode correlations within and across documents absent in random null models. We introduce and validate a generative model that explains the simultaneous emergence of all these patterns from simple rules. As a result, we find a connection between the bursty nature of rare words and the topical organization of texts and identify dynamic word ranking and memory across documents as key mechanisms explaining the non trivial organization of written text. Our research can have broad implications and practical applications in computer science, cognitive science and linguistics.
Comunicació escrita
Algorismes
Estadística
Written communication
Algorithms
Statistics
cc-by (c) Serrano Moral, Ma. Ángeles (María Ángeles) et al., 2009
http://creativecommons.org/licenses/by/3.0/es
Article
info:eu-repo/semantics/publishedVersion
Public Library of Science (PLoS)
         

Show full item record

Related documents

Other documents of the same author

Serrano Moral, Ma. Ángeles (María Ángeles); Krioukov, Dimitri; Boguñá, Marián
Serrano Moral, Ma. Ángeles (María Ángeles); Boguñá, Marián
Serrano Moral, Ma. Ángeles (María Ángeles); Boguñá, Marián
Boguñá, Marián; Serrano Moral, Ma. Ángeles (María Ángeles)
Serrano Moral, Ma. Ángeles (María Ángeles); Boguñá, Marián
 

Coordination

 

Supporters