title: Domain-specific language models and lexicons for tagging.
abstract: Accurate and reliable part-of-speech tagging is useful for many Natural Language Processing (NLP) tasks that form the foundation of NLP-based approaches to information retrieval and data mining. In general, large annotated corpora are necessary to achieve desired part-of-speech tagger accuracy. We show that a large annotated general-English corpus is not sufficient for building a part-of-speech tagger model adequate for tagging documents from the medical domain. However, adding a quite small domain-specific corpus to a large general-English one boosts performance to over 92% accuracy from 87% in our studies. We also suggest a number of characteristics to quantify the similarities between a training corpus and the test data. These results give guidance for creating an appropriate corpus for building a part-of-speech tagger model that gives satisfactory accuracy results on a new domain at a relatively small cost.
authors.affiliation: IBM, T.J. Watson Research Center, Hawthorne, NY 10532, USA. anni@us.ibm.com
authors.forename: Anni R
authors.initials: AR
authors.lastname: Coden
authors.forename: Serguei V
authors.initials: SV
authors.lastname: Pakhomov
authors.forename: Rie K
authors.initials: RK
authors.lastname: Ando
authors.forename: Patrick H
authors.initials: PH
authors.lastname: Duffy
authors.forename: Christopher G
authors.initials: CG
authors.lastname: Chute
mesh_terms: D006801:Humans; D007256:Information Systems; D007802:Language; D008390:Markov Chains; D009323:Natural Language Processing; D015203:Reproducibility of Results; D012660:Semantics; D017432:Unified Medical Language System