title: Comparing the Effectiveness of Alerts and Dynamically Annotated Visualizations (DAVs) in Improving Clinical Decision Making.
abstract: OBJECTIVE
  The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders.
  
  
  BACKGROUND
  Alerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making.
  
  
  METHODS
  Using a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance.
  
  
  RESULTS
  Fewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants.
  
  
  CONCLUSIONS
  DAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information.
  
  
  CONCLUSIONS
  Creating visualizations that are permanently displayed and vary in the strength of their guidance can mitigate the risk of system performance degradation due to incomplete or incorrect data. This interaction paradigm may be applicable for other settings with high false-alarm rates or where there is a need to reduce interruptions during decision making.
authors.affiliation: The Ohio State University, Columbus mike.rayo@cogenisys.com.
authors.forename: Michael F
authors.initials: MF
authors.lastname: Rayo
authors.affiliation: The Ohio State University, Columbus.
authors.forename: Nina
authors.initials: N
authors.lastname: Kowalczyk
authors.affiliation: The Ohio State University, Columbus.
authors.forename: Beth W
authors.initials: BW
authors.lastname: Liston
authors.affiliation: The Ohio State University, Columbus.
authors.forename: Elizabeth B-N
authors.initials: EB
authors.lastname: Sanders
authors.affiliation: The Ohio State University, Columbus.
authors.forename: Susan
authors.initials: S
authors.lastname: White
authors.affiliation: The Ohio State University, Columbus.
authors.forename: Emily S
authors.initials: ES
authors.lastname: Patterson
mesh_terms: D000328:Adult; D003657:Decision Making; D003661:Decision Support Techniques; D005260:Female; D006801:Humans; D008297:Male; D050316:Medical Order Entry Systems / Q000592:standards*; D010820:Physicians / Q000592:standards*; D055815:Young Adult
keywords: alarm; alert; alert fatigue; decision support system; electronic health record; evidence-based medicine; interruption; naturalistic decision making; visualization