title: Active safety monitoring of new medical products using electronic healthcare data:  selecting alerting rules.
abstract: BACKGROUND
  Active medical-product-safety surveillance systems are being developed to monitor many products and outcomes simultaneously in routinely collected longitudinal electronic healthcare data. These systems will rely on algorithms to generate alerts about potential safety concerns.
  
  
  METHODS
  We compared the performance of 5 classes of algorithms in simulated data using a sequential matched-cohort framework, and applied the results to 2 electronic healthcare databases to replicate monitoring of cerivastatin-induced rhabdomyolysis. We generated 600,000 simulated scenarios with varying expected event frequency in the unexposed, alerting threshold, and outcome risk in the exposed, and compared the alerting algorithms in each scenario type using an event-based performance metric.
  
  
  RESULTS
  We observed substantial variation in algorithm performance across the groups of scenarios. Relative performance varied by the event frequency and by user-defined preferences for sensitivity versus specificity. Type I error-based statistical testing procedures achieved higher event-based performance than other approaches in scenarios with few events, whereas statistical process control and disproportionality measures performed relatively better with frequent events. In the empirical data, we observed 6 cases of rhabdomyolysis among 4294 person-years of follow-up, with all events occurring among cerivastatin-treated patients. All selected algorithms generated alerts before the drug was withdrawn from the market.
  
  
  CONCLUSIONS
  For active medical-product-safety monitoring in a sequential matched cohort framework, no single algorithm performed best in all scenarios. Alerting algorithm selection should be tailored to particular features of a product-outcome pair, including the expected event frequencies and trade-offs between false-positive and false-negative alerting.
authors.affiliation: Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA 02120, USA. jgagne1@partners.org
authors.forename: Joshua J
authors.initials: JJ
authors.lastname: Gagne
authors.forename: Jeremy A
authors.initials: JA
authors.lastname: Rassen
authors.forename: Alexander M
authors.initials: AM
authors.lastname: Walker
authors.forename: Robert J
authors.initials: RJ
authors.lastname: Glynn
authors.forename: Sebastian
authors.initials: S
authors.lastname: Schneeweiss
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems / Q000458:organization & administration / Q000706:statistics & numerical data; D000465:Algorithms; D000924:Anticholesteremic Agents / Q000009:adverse effects; D057286:Electronic Health Records / Q000706:statistics & numerical data; D006801:Humans; D011358:Product Surveillance, Postmarketing / Q000379:methods; D011725:Pyridines / Q000009:adverse effects; D012206:Rhabdomyolysis / Q000139:chemically induced; D012680:Sensitivity and Specificity
chemical_list: D000924:Anticholesteremic Agents; D011725:Pyridines; C086276:cerivastatin