Determining the difficulty of Word Sense Disambiguation.
Automatic processing of biomedical documents is made difficult by the fact that many of the terms they contain are ambiguous. Word Sense Disambiguation (WSD) systems attempt to resolve these ambiguities and identify the correct meaning. However, the published literature on WSD systems for biomedical documents report considerable differences in performance for different terms. The development of WSD systems is often expensive with respect to acquiring the necessary training data. It would therefore be useful to be able to predict in advance which terms WSD systems are likely to perform well or badly on. This paper explores various methods for estimating the performance of WSD systems on a wide range of ambiguous biomedical terms (including ambiguous words/phrases and abbreviations). The methods include both supervised and unsupervised approaches. The supervised approaches make use of information from labeled training data while the unsupervised ones rely on the UMLS Metathesaurus. The approaches are evaluated by comparing their predictions about how difficult disambiguation will be for ambiguous terms against the output of two WSD systems. We find the supervised methods are the best predictors of WSD difficulty, but are limited by their dependence on labeled training data. The unsupervised methods all perform well in some situations and can be applied more widely.
Minnesota Supercomputing Institute, University of Minnesota, 117 Pleasant St SE, Minneapolis, MN 55455, USA. Electronic address: btmcinnes@gmail.com.
Bridget T
BT
McInnes
Natural Language Processing Group, Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello, Sheffield S1 4DP, United Kingdom. Electronic address: m.stevenson@dcs.shef.ac.uk.
Mark
M
Stevenson
D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D051188:Knowledge Bases; D007802:Language; D016239:MEDLINE; D008490:Medical Informatics; D015233:Models, Statistical; D009323:Natural Language Processing; D015203:Reproducibility of Results; D012660:Semantics; D017432:Unified Medical Language System; D018875:Vocabulary, Controlled
Ambiguity; Biomedical documents; NLP; Natural Language Processing; WSD; Word Sense Disambiguation