title: Mining Quality Phrases from Massive Text Corpora.
abstract: Text data are ubiquitous and play an essential role in big data applications. However, text data are mostly unstructured. Transforming unstructured text into structured units (e.g., semantically meaningful phrases) will substantially reduce semantic ambiguity and enhance the power and efficiency at manipulating such data using database technology. Thus mining quality phrases is a critical research problem in the field of databases. In this paper, we propose a new framework that extracts quality phrases from text corpora integrated with phrasal segmentation. The framework requires only limited training but the quality of phrases so generated is close to human judgment. Moreover, the method is scalable: both computation time and required space grow linearly as corpus size increases. Our experiments on large text corpora demonstrate the quality and efficiency of the new method.
authors.affiliation: Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
authors.forename: Jialu
authors.initials: J
authors.lastname: Liu
authors.affiliation: Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
authors.forename: Jingbo
authors.initials: J
authors.lastname: Shang
authors.affiliation: Microsoft Research, Redmond, WA, USA.
authors.forename: Chi
authors.initials: C
authors.lastname: Wang
authors.affiliation: Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
authors.forename: Xiang
authors.initials: X
authors.lastname: Ren
authors.affiliation: Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
authors.forename: Jiawei
authors.initials: J
authors.lastname: Han