title: Automatic learning of the morphology of medical language using information compression.
abstract: Conversion of free-text strings in a natural language to a standard representation (codes) is an important reoccurring problem in biomedical informatics. Determining the content of a string involves identifying its meaningful constituents (morphemes). One current method of identifying these constituents is to look them up in a preexisting table (lexicon). Manual construction of lexicons and grammars in complex domains such as biomedicine is extremely laborious. As an alternative to the lexico-grammatical approach, we introduce a segmentation algorithm that automatically learns lexical and structural preferences from corpora via information compression. The method is based on the Minimum Description Length (MDL) principle from classic information theory.
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA.
authors.forename: Shamim Ara
authors.initials: SA
authors.lastname: Mollah
authors.forename: Stephen B
authors.initials: SB
authors.lastname: Johnson
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D044962:Data Compression; D007257:Information Theory; D009323:Natural Language Processing; D017432:Unified Medical Language System