Automatic learning of the morphology of medical language using information compression.
Conversion of free-text strings in a natural language to a standard representation (codes) is an important reoccurring problem in biomedical informatics. Determining the content of a string involves identifying its meaningful constituents (morphemes). One current method of identifying these constituents is to look them up in a preexisting table (lexicon). Manual construction of lexicons and grammars in complex domains such as biomedicine is extremely laborious. As an alternative to the lexico-grammatical approach, we introduce a segmentation algorithm that automatically learns lexical and structural preferences from corpora via information compression. The method is based on the Minimum Description Length (MDL) principle from classic information theory.
Mollah
Shamim Ara
SA
Department of Biomedical Informatics, Columbia University, New York, NY, USA.
Johnson
Stephen B
SB
D000465:Algorithms; D001185:Artificial Intelligence; D044962:Data Compression; D007257:Information Theory; D009323:Natural Language Processing; D017432:Unified Medical Language System