Improving textual medication extraction using combined conditional random fields and rule-based systems.
OBJECTIVE
In the i2b2 Medication Extraction Challenge, medication names together with details of their administration were to be extracted from medical discharge summaries.


METHODS
The task of the challenge was decomposed into three pipelined components: named entity identification, context-aware filtering and relation extraction. For named entity identification, first a rule-based (RB) method that was used in our overall fifth place-ranked solution at the challenge was investigated. Second, a conditional random fields (CRF) approach is presented for named entity identification (NEI) developed after the completion of the challenge. The CRF models are trained on the 17 ground truth documents, the output of the rule-based NEI component on all documents, a larger but potentially inaccurate training dataset. For both NEI approaches their effect on relation extraction performance was investigated. The filtering and relation extraction components are both rule-based.


METHODS
In addition to the official entry level evaluation of the challenge, entity level analysis is also provided.


RESULTS
On the test data an entry level F(1)-score of 80% was achieved for exact matching and 81% for inexact matching with the RB-NEI component. The CRF produces a significantly weaker result, but CRF outperforms the rule-based model with 81% exact and 82% inexact F(1)-score (p<0.02).


CONCLUSIONS
This study shows that a simple rule-based method is on a par with more complicated machine learners; CRF models can benefit from the addition of the potentially inaccurate training data, when only very few training documents are available. Such training data could be generated using the outputs of rule-based methods.
Tikk
Domonkos
D
Department of Telecommunications and Media Informatics, Budapest University of Technology and Economics, Budapest, Hungary.
Solt
Illés
I
D057286:Electronic Health Records; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D009323:Natural Language Processing; D004364:Pharmaceutical Preparations / Q000008:administration & dosage; D012985:Software Design
D004364:Pharmaceutical Preparations
J Am Med Inform Assoc. 1998 Jan-Feb;5(1):62-75
9452986
AMIA Annu Symp Proc. 2008 Nov 06;:237-41
18999147
BMC Bioinformatics. 2005;6 Suppl 1:S6
15960840
Drug Discov Today. 2005 Mar 15;10(6):439-45
15808823
BMC Bioinformatics. 2008 Apr 11;9 Suppl 3:S10
18426545
Proc AMIA Annu Fall Symp. 1996;:388-92
8947694
Int J Med Inform. 2009 Apr;78(4):284-91
18838293
Genome Biol. 2008;9 Suppl 2:S14
18834492
AMIA Annu Symp Proc. 2008 Nov 06;:293-7
18999272
Brief Bioinform. 2005 Mar;6(1):57-71
15826357
J Am Med Inform Assoc. 2009 Jul-Aug;16(4):580-4
19390101
Bioinformatics. 2006 Oct 1;22(19):2444-5
16870931
J Am Med Inform Assoc. 2007 Sep-Oct;14(5):550-63
17600094
J Am Med Inform Assoc. 2010 Sep-Oct;17(5):514-8
20819854
J Biomed Inform. 2008 Apr;41(2):393-407
18207462
J Am Med Inform Assoc. 2007 Sep-Oct;14(5):564-73
17600096