title: Accuracy of using automated methods for detecting adverse events from electronic health record data: a research protocol.
abstract: BACKGROUND
  Adverse events are associated with significant morbidity, mortality and cost in hospitalized patients. Measuring adverse events is necessary for quality improvement, but current detection methods are inaccurate, untimely and expensive. The advent of electronic health records and the development of automated methods for encoding and classifying electronic narrative data, such as natural language processing, offer an opportunity to identify potentially better methods. The objective of this study is to determine the accuracy of using automated methods for detecting three highly prevalent adverse events: a) hospital-acquired pneumonia, b) catheter-associated bloodstream infections, and c) in-hospital falls.
  
  
  METHODS
  This validation study will be conducted at two large Canadian academic health centres: the McGill University Health Centre (MUHC) and The Ottawa Hospital (TOH). The study population consists of all medical, surgical and intensive care unit patients admitted to these centres between 2008 and 2014. An automated detection algorithm will be developed and validated for each of the three adverse events using electronic data extracted from multiple clinical databases. A random sample of MUHC patients will be used to develop the automated detection algorithms (cohort 1, development set). The accuracy of these algorithms will be assessed using chart review as the reference standard. Then, receiver operating characteristic curves will be used to identify optimal cut points for each of the data sources. Multivariate logistic regression and the areas under curve (AUC) will be used to identify the optimal combination of data sources that maximize the accuracy of adverse event detection. The most accurate algorithms will then be validated on a second random sample of MUHC patients (cohort 1, validation set), and accuracy will be measured using chart review as the reference standard. The most accurate algorithms validated at the MUHC will then be applied to TOH data (cohort 2), and their accuracy will be assessed using a reference standard assessment of the medical chart.
  
  
  CONCLUSIONS
  There is a need for more accurate, timely and efficient measures of adverse events in acute care hospitals. This is a critical requirement for evaluating the effectiveness of preventive interventions and for tracking progress in patient safety through time.
authors.affiliation: Ingram School of Nursing, Faculty of Medicine, McGill University, Wilson Hall, 3506 University Street, Montreal, QC, H3A 2A7, Canada. Christian.Rochefort@mcgill.ca.|McGill Clinical and Health Informatics Research Group, McGill University, 1140, Pine Avenue West, Montreal, QC, H3A 1A3, Canada. Christian.Rochefort@mcgill.ca.|Department of Epidemiology, Biostatics and Occupational Health, Faculty of Medicine, McGill University, Purvis Hall, 1020 Pine Avenue West, Montreal, QC, H3A 1A2, Canada. Christian.Rochefort@mcgill.ca.
authors.forename: Christian M
authors.initials: CM
authors.lastname: Rochefort
authors.affiliation: McGill Clinical and Health Informatics Research Group, McGill University, 1140, Pine Avenue West, Montreal, QC, H3A 1A3, Canada. david.buckeridge@mcgill.ca.|Department of Epidemiology, Biostatics and Occupational Health, Faculty of Medicine, McGill University, Purvis Hall, 1020 Pine Avenue West, Montreal, QC, H3A 1A2, Canada. david.buckeridge@mcgill.ca.
authors.forename: David L
authors.initials: DL
authors.lastname: Buckeridge
authors.affiliation: Ottawa Hospital Research Institute, Ottawa, ON, Canada. aforster@ohri.ca.|The Ottawa Hospital, 725 Parkdale Ave, Ottawa, ON, K1Y 4E9, Canada. aforster@ohri.ca.
authors.forename: Alan J
authors.initials: AJ
authors.lastname: Forster
mesh_terms: D000058:Accidental Falls / Q000706:statistics & numerical data*; D000465:Algorithms; D055499:Catheter-Related Infections / Q000453:epidemiology*; D003428:Cross Infection / Q000453:epidemiology*; D057225:Data Mining / Q000379:methods*; D057286:Electronic Health Records / Q000592:standards / Q000706:statistics & numerical data; D006801:Humans; D007297:Inpatients / Q000706:statistics & numerical data; D011014:Pneumonia / Q000453:epidemiology; D018401:Sample Size