AskHERMES: An online question answering system for complex clinical questions.
OBJECTIVE
Clinical questions are often long and complex and take many forms. We have built a clinical question answering system named AskHERMES to perform robust semantic analysis on complex clinical questions and output question-focused extractive summaries as answers.


METHODS
This paper describes the system architecture and a preliminary evaluation of AskHERMES, which implements innovative approaches in question analysis, summarization, and answer presentation. Five types of resources were indexed in this system: MEDLINE abstracts, PubMed Central full-text articles, eMedicine documents, clinical guidelines and Wikipedia articles.


METHODS
We compared the AskHERMES system with Google (Google and Google Scholar) and UpToDate and asked physicians to score the three systems by ease of use, quality of answer, time spent, and overall performance.


RESULTS
AskHERMES allows physicians to enter a question in a natural way with minimal query formulation and allows physicians to efficiently navigate among all the answer sentences to quickly meet their information needs. In contrast, physicians need to formulate queries to search for information in Google and UpToDate. The development of the AskHERMES system is still at an early stage, and the knowledge resource is limited compared with Google or UpToDate. Nevertheless, the evaluation results show that AskHERMES' performance is comparable to the other systems. In particular, when answering complex clinical questions, it demonstrates the potential to outperform both Google and UpToDate systems.


CONCLUSIONS
AskHERMES, available at http://www.AskHERMES.org, has the potential to help physicians practice evidence-based medicine and improve the quality of patient care.
Department of Health Sciences, University of Wisconsin-Milwaukee, 2400 E. Hartford Avenue, Milwaukee, WI 53211, USA.
YongGang
Y
Cao
Feifan
F
Liu
Pippa
P
Simpson
Lamont
L
Antieau
Andrew
A
Bennett
James J
JJ
Cimino
John
J
Ely
Hong
H
Yu
D000465:Algorithms; D015510:Clinical Medicine; D016208:Databases, Factual; D005103:Expert Systems; D016247:Information Storage and Retrieval / Q000379:methods; D009323:Natural Language Processing; D009862:Online Systems; D012984:Software
Bull Med Libr Assoc. 1993 Apr;81(2):195-206
8472005
Aust Fam Physician. 2008 Oct;37(10):878-82
19002313
J Fam Pract. 1997 Nov;45(5):382-8
9374962
J Biomed Inform. 2007 Jun;40(3):236-51
17462961
BMJ. 2000 Dec 16;321(7275):1511-5
11118181
J Am Med Inform Assoc. 2009 Jan-Feb;16(1):140-2
18952943
AMIA Annu Symp Proc. 2006;:902
17238521
AMIA Annu Symp Proc. 2006;:469-73
17238385
J Med Libr Assoc. 2002 Oct;90(4):370-9
12398243
AMIA Annu Symp Proc. 2008 Nov 06;:116-20
18999097
PLoS Biol. 2004 Nov;2(11):e309
15383839
Proc AMIA Symp. 2002;:170-4
12463809
Summit Transl Bioinform. 2009 Mar 01;2009:143-7
21347188
BMJ. 1997 Jun 28;314(7098):1879-81
9224133
Proc AMIA Symp. 2001;:17-21
11825149
J Med Internet Res. 2008 Oct 03;10(4):e29
18926978
Proc AMIA Annu Fall Symp. 1997;:528-32
9357682
Artif Intell Med. 2005 Feb;33(2):179-98
15811784
JAMA. 1998 Feb 25;279(8):611-4
9486757
Ann Pharmacother. 2009 Mar;43(3):478-84
19261965
Mil Med. 2009 Mar;174(3):259-64
19354089
J Am Med Inform Assoc. 2002 May-Jun;9(3):283-93
11971889
J Am Med Inform Assoc. 2008 Nov-Dec;15(6):752-9
18755999
Stud Health Technol Inform. 2004;107(Pt 1):306-10
15360824
Acad Radiol. 2007 Sep;14(9):1113-20
17707320
Health Promot Int. 2003 Dec;18(4):381-6
14695369
BMJ. 1996 Oct 26;313(7064):1062-8
8898602
AMIA Annu Symp Proc. 2003;:906
14728412
AMIA Annu Symp Proc. 2006;:359-63
17238363
J Am Med Inform Assoc. 2007 Nov-Dec;14(6):772-80
17712086
J Am Coll Dent. 1999 Summer;66(2):43-5
10506806
Proc AMIA Symp. 2002;:722-6
12463919
Bull Med Libr Assoc. 1993 Apr;81(2):170-7
8472002
AMIA Annu Symp Proc. 2006;:151-5
17238321
J Fam Pract. 1992 Sep;35(3):265-9
1517722
Acad Med. 2007 Mar;82(3):298-303
17327723
AMIA Annu Symp Proc. 2008 Nov 06;:96-100
18999100
BMJ. 2000 Aug 12;321(7258):429-32
10938054
Methods Inf Med. 1990 Jan;29(1):23-9
2407930
BMJ. 2002 Mar 9;324(7337):557-8
11884303
Proc AMIA Annu Fall Symp. 1997;:480-4
9357672
BMC Bioinformatics. 2006 Mar 16;7:140
16539745
BMJ. 1999 Aug 7;319(7206):358-61
10435959
BMJ. 2006 Dec 2;333(7579):1143-5
17098763
AMIA Annu Symp Proc. 2003;:175-9
14728157
JAMA. 1997 Apr 16;277(15):1244-5
9103351
J Am Med Inform Assoc. 2005 Mar-Apr;12(2):217-24
15561792
Ann Intern Med. 1991 Apr 1;114(7):576-81
2001091
J Am Med Inform Assoc. 2007 May-Jun;14(3):253-63
17329729
Ann Intern Med. 1985 Oct;103(4):596-9
4037559
BMC Med Educ. 2008 Jul 14;8:39
18625038
AMIA Annu Symp Proc. 2003;:815
14728320
AMIA Annu Symp Proc. 2006;:834-8
17238458
Arch Fam Med. 2000 Jun;9(6):541-7
10862217
AMIA Annu Symp Proc. 2005;:226-30
16779035
J Med Internet Res. 2005 Dec 30;7(5):e59
16403723
Pac Symp Biocomput. 2007;:328-39
17990503
J Am Med Inform Assoc. 2009 May-Jun;16(3):387-94
19261939
J Gen Intern Med. 1997 Apr;12 Suppl 2:S5-14
9127238
AMIA Annu Symp Proc. 2008 Nov 06;:1203-4
18998777
Health Educ Res. 2001 Dec;16(6):671-92
11780707
Pediatrics. 2004 Jan;113(1 Pt 1):64-9
14702450