title: The Number Needed to Remind: a Measure for Assessing CDS Effectiveness.
abstract: BACKGROUND
  Clinical decision support (CDS) is associated with improvement in quality and efficiency in healthcare delivery. The appropriate way to evaluate its effectiveness remains uncertain.
  
  
  METHODS
  We analyzed data from our electronic health record (EHR) measuring the display frequency of eight reminders for Coronary Artery disease and Type 2 Diabetes and their associated performance according to a predefined methodology. We propose two key performance indicators to measure their impact on a target population: the reminder performance (RP), and the number needed to remind (NNR), to evaluate the impact that Clinical decision support reminders have on the adherence to guideline derived CDS interventions on the entire patient population, and individual providers receiving the interventions.
  
  
  RESULTS
  Data were available for 116,027 patients and a total of 1,982,735 reminders were displayed to a subset of 65,516 patients during the study period from January 1 to December 31, 2010. The evaluation framework assessed provider acknowledgement of the CDS intervention, and the presence of the expected performance event while accounting for patients' exposure to the CDS reminders. The total RP was 2.7% while the average NNR was 3.1 for all the reminders under study.
  
  
  CONCLUSIONS
  The proposed framework to asses of CDS performance provides a novel approach to improve the design and evaluation of CDS interventions. The application of this methodology represents an indicator to understand the impact of CDS interventions and subsequent patient outcomes. Further research is required to evaluate the impact of these systems on the quality of care.
authors.affiliation: Clinical Informatics Research and Development, Partners Healthcare Systems, Wellesley, MA ; Harvard Medical School, Boston, MA ; Massachusetts General Hospital, Boston, MA.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Einbinder
authors.affiliation: Clinical Informatics Research and Development, Partners Healthcare Systems, Wellesley, MA ; German Clinic, Santiago, Chile.
authors.forename: Esteban
authors.initials: E
authors.lastname: Hebel
authors.affiliation: Clinical Informatics Research and Development, Partners Healthcare Systems, Wellesley, MA ; Harvard Medical School, Boston, MA ; Massachusetts General Hospital, Boston, MA.
authors.forename: Adam
authors.initials: A
authors.lastname: Wright
authors.affiliation: Massachusetts General Hospital, Boston, MA.
authors.forename: Morgan
authors.initials: M
authors.lastname: Panzenhagen
authors.affiliation: Clinical Informatics Research and Development, Partners Healthcare Systems, Wellesley, MA ; Harvard Medical School, Boston, MA ; Massachusetts General Hospital, Boston, MA.
authors.forename: Blackford
authors.initials: B
authors.lastname: Middleton
mesh_terms: D020000:Decision Support Systems, Clinical; D057286:Electronic Health Records; D006442:Glycated Hemoglobin; D006801:Humans; D017010:Reminder Systems; D014584:User-Computer Interface
chemical_list: D006442:Glycated Hemoglobin; C517652:hemoglobin A1c protein, human