title: Development and empirical user-centered evaluation of semantically-based query recommendation for an electronic health record search engine.
abstract: The utility of biomedical information retrieval environments can be severely limited when users lack expertise in constructing effective search queries. To address this issue, we developed a computer-based query recommendation algorithm that suggests semantically interchangeable terms based on an initial user-entered query. In this study, we assessed the value of this approach, which has broad applicability in biomedical information retrieval, by demonstrating its application as part of a search engine that facilitates retrieval of information from electronic health records (EHRs).
  
  
  
  The query recommendation algorithm utilizes MetaMap to identify medical concepts from search queries and indexed EHR documents. Synonym variants from UMLS are used to expand the concepts along with a synonym set curated from historical EHR search logs. The empirical study involved 33 clinicians and staff who evaluated the system through a set of simulated EHR search tasks. User acceptance was assessed using the widely used technology acceptance model.
  
  
  
  The search engine's performance was rated consistently higher with the query recommendation feature turned on vs. off. The relevance of computer-recommended search terms was also rated high, and in most cases the participants had not thought of these terms on their own. The questions on perceived usefulness and perceived ease of use received overwhelmingly positive responses. A vast majority of the participants wanted the query recommendation feature to be available to assist in their day-to-day EHR search tasks.
  
  
  
  Challenges persist for users to construct effective search queries when retrieving information from biomedical documents including those from EHRs. This study demonstrates that semantically-based query recommendation is a viable solution to addressing this challenge.
authors.affiliation: Department of Pediatrics, University of Michigan Medical School, 5312 CC, SPC 5940, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA. Electronic address: hanauer@med.umich.edu.
authors.forename: David A
authors.initials: DA
authors.lastname: Hanauer
authors.affiliation: School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA; Department of Pediatrics, University of Michigan Medical School, 5312 CC, SPC 5940, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA. Electronic address: tzuyu@umich.edu.
authors.forename: Danny T Y
authors.initials: DT
authors.lastname: Wu
authors.affiliation: School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA. Electronic address: yangle@umich.edu.
authors.forename: Lei
authors.initials: L
authors.lastname: Yang
authors.affiliation: School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA; Department of Electrical Engineering and Computer Science, University of Michigan, 2260 Hayward Street, Ann Arbor, MI 48109, USA. Electronic address: qmei@umich.edu.
authors.forename: Qiaozhu
authors.initials: Q
authors.lastname: Mei
authors.affiliation: Department of Health Management and Policy, School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109, USA. Electronic address: kbmurko@umich.edu.
authors.forename: Katherine B
authors.initials: KB
authors.lastname: Murkowski-Steffy
authors.affiliation: Department of Learning Health Sciences, University of Michigan Medical School, 1111 East Catherine Street, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA. Electronic address: vgvinodv@med.umich.edu.
authors.forename: V G Vinod
authors.initials: VG
authors.lastname: Vydiswaran
authors.affiliation: Department of Health Management and Policy, School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA. Electronic address: kzheng@umich.edu.
authors.forename: Kai
authors.initials: K
authors.lastname: Zheng
mesh_terms: D000465:Algorithms; D057286:Electronic Health Records; D006801:Humans; D016247:Information Storage and Retrieval; D009323:Natural Language Processing; D057386:Search Engine; D012660:Semantics
keywords: Electronic health records (E05.318.308.940.968.625.500); Information retrieval systems (L01.700.508.300); Query expansion; Query recommendation; Search engine (L01.470.875); Unified Medical Language System (L01.453.245.945.800)