title: Development of an algorithm to assess appropriateness of overriding alerts for nonformulary medications in a computerized prescriber-order-entry system.
abstract: OBJECTIVE
  An algorithm for assessing the appropriateness of physician overrides of clinical decision support alerts triggered by nonformulary medication (NFM) requests is described.
  
  
  METHODS
  Data on a random sample of 5000 NFM alert overrides at Brigham and Women's Hospital over a four-year period (2009-12) were extracted from the hospital's computerized prescriber-order-entry (CPOE) system. Through an iterative process, a scheme for categorizing the reasons given by prescribers for alert overrides was developed. A pharmacist and a physician used the categorization scheme to classify and group alert override reasons, and the resultant data guided the development of an algorithm for assessing alert overrides.
  
  
  RESULTS
  In free-text comments written in response to NFM alerts, prescribers provided more than 1150 unique reasons to justify formulary deviation. The compiled reasons were analyzed and grouped into nine categories through the iterative process, with a high degree of interrater agreement (κ = 0.989; 95% confidence interval, 0.985-0.992). An initially developed 30-item "NFM alert override appropriateness algorithm" was simplified to create an 8-question algorithm that was presented to an interdisciplinary team for evaluation, with subsequent refinements for enhanced clinical creditability. The final algorithm can be used by researchers and formulary managers to develop strategies for limiting NFM alert overrides and to avoid the labor-intensive task of creating appropriateness criteria for each NFM.
  
  
  CONCLUSIONS
  A multistep process was used to develop a generalized algorithm for categorizing the appropriateness of reasons given for NFM alert overrides in a CPOE system.
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH. qher@partners.org.
authors.forename: Qoua L
authors.initials: QL
authors.lastname: Her
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Diane L
authors.initials: DL
authors.lastname: Seger
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Mary G
authors.initials: MG
authors.lastname: Amato
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Patrick E
authors.initials: PE
authors.lastname: Beeler
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Olivia
authors.initials: O
authors.lastname: Dalleur
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Sarah P
authors.initials: SP
authors.lastname: Slight
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: Patricia C
authors.initials: PC
authors.lastname: Dykes
authors.affiliation: Qoua L. Her, Pharm.D., M.S., M.Sc., is Pharmacy Informatics and Outcomes Research Fellow, MCPHS University School of Pharmacy-Boston and Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital (BWH), Boston, MA. Diane L. Seger, B.S.Pharm., is Senior Pharmaco-Informatics Specialist, Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Inc., Wellesley, MA. Mary G. Amato, Pharm.D., M.P.H., is Associate Professor of Pharmacy Practice, MCPHS University, Boston. Patrick E. Beeler, M.D., is Postdoctoral Research Fellow, Division of General Internal Medicine and Primary Care, BWH, and Research Fellow, Harvard Medical School, Boston. Olivia Dalleur, M.Pharm., Ph.D., is Research Associate in Medicine, Division of General Internal Medicine and Primary Care, BWH, and Assistant Professor in Clinical Pharmacy, Louvain Drug Research Institute, Clinical Pharmacy Research Group, Cliniques, Universitaires Saint-Luc, Université catholique de Louvain, Brussels, Belgium. Sarah P. Slight, M.Pharm., Ph.D., P.G.Dip., is Senior Lecturer/Associate Professor of Pharmacy Practice, Wolfson Research Institute, School of Medicine, Pharmacy and Health, Durham University, Queen's Campus, Stockton-on-Tees, England, and Visiting Research Scholar, Division of General Internal Medicine and Primary Care, BWH. Patricia C. Dykes, Ph.D., RN, FACMI, is Program Director, Center for Patient Safety, Research and Practice, BWH. David W. Bates, M.D., M.Sc., is Senior Vice President for Quality and Safety and Chief Quality Officer, Division of General Internal Medicine and Primary Care, BWH.
authors.forename: David W
authors.initials: DW
authors.lastname: Bates
mesh_terms: D000465:Algorithms; D020000:Decision Support Systems, Clinical / Q000592:standards; D004360:Drug Therapy, Computer-Assisted / Q000379:methods / Q000592:standards*; D005572:Formularies, Hospital as Topic / Q000592:standards*; D006801:Humans; D050316:Medical Order Entry Systems / Q000592:standards*; D008508:Medication Errors / Q000517:prevention & control*