title: Usability Evaluation of a Clinical Decision Support System for Geriatric ED Pain Treatment.
abstract: BACKGROUND
  Older adults are at risk for inadequate emergency department (ED) pain care. Unrelieved acute pain is associated with poor outcomes. Clinical decision support systems (CDSS) hold promise to improve patient care, but CDSS quality varies widely, particularly when usability evaluation is not employed.
  
  
  OBJECTIVE
  To conduct an iterative usability and redesign process of a novel geriatric abdominal pain care CDSS. We hypothesized this process would result in the creation of more usable and favorable pain care interventions.
  
  
  METHODS
  Thirteen emergency physicians familiar with the Electronic Health Record (EHR) in use at the study site were recruited. Over a 10-week period, 17 1-hour usability test sessions were conducted across 3 rounds of testing. Participants were given 3 patient scenarios and provided simulated clinical care using the EHR, while interacting with the CDSS interventions. Quantitative System Usability Scores (SUS), favorability scores and qualitative narrative feedback were collected for each session. Using a multi-step review process by an interdisciplinary team, positive and negative usability issues in effectiveness, efficiency, and satisfaction were considered, prioritized and incorporated in the iterative redesign process of the CDSS. Video analysis was used to determine the appropriateness of the CDS appearances during simulated clinical care.
  
  
  RESULTS
  Over the 3 rounds of usability evaluations and subsequent redesign processes, mean SUS progressively improved from 74.8 to 81.2 to 88.9; mean favorability scores improved from 3.23 to 4.29 (1 worst, 5 best). Video analysis revealed that, in the course of the iterative redesign processes, rates of physicians' acknowledgment of CDS interventions increased, however most rates of desired actions by physicians (such as more frequent pain score updates) decreased.
  
  
  CONCLUSIONS
  The iterative usability redesign process was instrumental in improving the usability of the CDSS; if implemented in practice, it could improve geriatric pain care. The usability evaluation process led to improved acknowledgement and favorability. Incorporating usability testing when designing CDSS interventions for studies may be effective to enhance clinician use.
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Nicholas
authors.initials: N
authors.lastname: Genes
authors.affiliation: Department of Health Management & Informatics, University of Missouri School of Medicine, Columbia, MO; Informatics Institute, University of Missouri, Columbia, MO.
authors.forename: Min Soon
authors.initials: MS
authors.lastname: Kim
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Frederick L
authors.initials: FL
authors.lastname: Thum
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Laura
authors.initials: L
authors.lastname: Rivera
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Rosemary
authors.initials: R
authors.lastname: Beato
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Carolyn
authors.initials: C
authors.lastname: Song
authors.affiliation: Information Technology, Mount Sinai Health System , New York, NY.
authors.forename: Jared
authors.initials: J
authors.lastname: Soriano
authors.affiliation: Information Technology, Mount Sinai Health System, New York, NY; Division of General Internal Medicine, Icahn School of Medicine at Mount Sinai, New York, NY.
authors.forename: Joseph
authors.initials: J
authors.lastname: Kannry
authors.affiliation: Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai , New York, NY.
authors.forename: Kevin
authors.initials: K
authors.lastname: Baumlin
authors.affiliation: Brookdale Department of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; Geriatric Research, Education and Clinical Center, James J Peters VAMC, Bronx, NY.
authors.forename: Ula
authors.initials: U
authors.lastname: Hwang
mesh_terms: D000368:Aged; D020000:Decision Support Systems, Clinical / Q000706:statistics & numerical data*; D004636:Emergency Service, Hospital; D005853:Geriatrics; D006801:Humans; D059408:Pain Management; D010820:Physicians / Q000706:statistics & numerical data
keywords: Clinical decision support systems; acute pain; aged; computerized; decision support techniques; man-machine systems; medical record systems