Automatic summarization of MEDLINE citations for evidence-based medical treatment: a topic-oriented evaluation.
As the number of electronic biomedical textual resources increases, it becomes harder for physicians to find useful answers at the point of care. Information retrieval applications provide access to databases; however, little research has been done on using automatic summarization to help navigate the documents returned by these systems. After presenting a semantic abstraction automatic summarization system for MEDLINE citations, we concentrate on evaluating its ability to identify useful drug interventions for 53 diseases. The evaluation methodology uses existing sources of evidence-based medicine as surrogates for a physician-annotated reference standard. Mean average precision (MAP) and a clinical usefulness score developed for this study were computed as performance metrics. The automatic summarization system significantly outperformed the baseline in both metrics. The MAP gain was 0.17 (p<0.01) and the increase in the overall score of clinical usefulness was 0.39 (p<0.05).
Fiszman
Marcelo
M
National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bldg 38A, Rm B1N-28J, Bethesda, MD 20894, USA. fiszmanm@mail.nih.gov
Demner-Fushman
Dina
D
Kilicoglu
Halil
H
Rindflesch
Thomas C
TC
D001185:Artificial Intelligence; D004194:Disease; D004358:Drug Therapy; D001330:Electronic Data Processing / Q000379:methods*; D019317:Evidence-Based Medicine / Q000379:methods*; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods; D020407:Internet; D016239:MEDLINE; D008490:Medical Informatics / Q000379:methods*; D009323:Natural Language Processing; D012660:Semantics; D014584:User-Computer Interface; D018875:Vocabulary, Controlled