Combining an expert-based medical entity recognizer to a machine-learning system: methods and a case study.
Medical entity recognition is currently generally performed by data-driven methods based on supervised machine learning. Expert-based systems, where linguistic and domain expertise are directly provided to the system are often combined with data-driven systems. We present here a case study where an existing expert-based medical entity recognition system, Ogmios, is combined with a data-driven system, Caramba, based on a linear-chain Conditional Random Field (CRF) classifier. Our case study specifically highlights the risk of overfitting incurred by an expert-based system. We observe that it prevents the combination of the 2 systems from obtaining improvements in precision, recall, or F-measure, and analyze the underlying mechanisms through a post-hoc feature-level analysis. Wrapping the expert-based system alone as attributes input to a CRF classifier does boost its F-measure from 0.603 to 0.710, bringing it on par with the data-driven system. The generalization of this method remains to be further investigated.
LIMSI-CNRS, Orsay, France.
Pierre
P
Zweigenbaum
Thomas
T
Lavergne
Natalia
N
Grabar
Thierry
T
Hamon
Sophie
S
Rosset
Cyril
C
Grouin
hybrid methods; information extraction; machine learning; medical records; natural language processing; overfitting