title: Incorporating linguistic knowledge for learning distributed word representations.
abstract: Combined with neural language models, distributed word representations achieve significant advantages in computational linguistics and text mining. Most existing models estimate distributed word vectors from large-scale data in an unsupervised fashion, which, however, do not take rich linguistic knowledge into consideration. Linguistic knowledge can be represented as either link-based knowledge or preference-based knowledge, and we propose knowledge regularized word representation models (KRWR) to incorporate these prior knowledge for learning distributed word representations. Experiment results demonstrate that our estimated word representation achieves better performance in task of semantic relatedness ranking. This indicates that our methods can efficiently encode both prior knowledge from knowledge bases and statistical knowledge from large-scale text corpora into a unified word representation model, which will benefit many tasks in text mining.
authors: State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China.
authors: Yan
authors: Y
authors: Wang
authors: State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China.
authors: Zhiyuan
authors: Z
authors: Liu
authors: State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China; Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China.
authors: Maosong
authors: M
authors: Sun
mesh_terms: D000465:Algorithms; D057225:Data Mining; D006801:Humans; D051188:Knowledge Bases; D007802:Language; D008037:Linguistics / Q000379:methods*; D008962:Models, Theoretical; D009323:Natural Language Processing; D012660:Semantics; D014706:Verbal Learning