title: Evaluating performance of electronic healthcare records and spontaneous reporting data in drug safety signal detection.
abstract: BACKGROUND
  Electronic reporting and processing of suspected adverse drug reactions (ADRs) is increasing and has facilitated automated screening procedures. It is crucial for healthcare professionals to understand the nature and proper use of data available in pharmacovigilance practice.
  
  
  OBJECTIVE
  To (a) compare performance of EU-ADR [electronic healthcare record (EHR) exemplar] and FAERS [spontaneous reporting system (SRS) exemplar] databases in detecting signals using "positive" and "negative" drug-event reference sets; and (b) evaluate the impact of timing bias on sensitivity thresholds by comparing all data to data restricted to the time before a warning/regulatory action.
  
  
  METHODS
  Ten events with known positive and negative reference sets were selected. Signals were identified when respective statistics exceeded defined thresholds. Main outcome measure Performance metrics, including sensitivity, specificity, positive predictive value and accuracy were calculated. In addition, the effect of regulatory action on the performance of signal detection in each data source was evaluated.
  
  
  RESULTS
  The sensitivity for detecting signals in EHR data varied depending on the nature of the adverse events and increased substantially if the analyses were restricted to the period preceding the first regulatory action. Across all events, using data from all years, a sensitivity of 45-73 % was observed for EU-ADR and 77 % for FAERS. The specificity was high and similar for EU-ADR (82-96 %) and FAERS (98 %). EU-ADR data showed range of PPV (78-91 %) and accuracy (78-72 %) and FAERS data yielded a PPV of 97 % with 88 % accuracy.
  
  
  CONCLUSIONS
  Using all cumulative data, signal detection in SRS data achieved higher specificity and sensitivity than EHR data. However, when data were restricted to time prior to a regulatory action, performance characteristics changed in a manner consistent with both the type of data and nature of the ADR. Further research focusing on prospective validation of is necessary to learn more about the performance and utility of these databases in modern pharmacovigilance practice.
authors.affiliation: Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands, v.patadia@erasmusmc.nl.
authors.forename: Vaishali K
authors.initials: VK
authors.lastname: Patadia
authors.forename: Martijn J
authors.initials: MJ
authors.lastname: Schuemie
authors.forename: Preciosa
authors.initials: P
authors.lastname: Coloma
authors.forename: Ron
authors.initials: R
authors.lastname: Herings
authors.forename: Johan
authors.initials: J
authors.lastname: van der Lei
authors.forename: Sabine
authors.initials: S
authors.lastname: Straus
authors.forename: Miriam
authors.initials: M
authors.lastname: Sturkenboom
authors.forename: Gianluca
authors.initials: G
authors.lastname: Trifirò
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems / Q000592:standards*; D016208:Databases, Factual / Q000592:standards*; D057286:Electronic Health Records / Q000592:standards*; D006801:Humans; D011446:Prospective Studies; D012189:Retrospective Studies