title: Are all quantitative postmarketing signal detection methods equal? Performance characteristics of logistic regression and Multi-item Gamma Poisson Shrinker.
abstract: OBJECTIVE
  The detection of safety signals with medicines is an essential activity to protect public health. Despite widespread acceptance, it is unclear whether recently applied statistical algorithms provide enhanced performance characteristics when compared with traditional systems. Novartis has adopted a novel system for automated signal detection on the basis of disproportionality methods within a safety data mining application (Empirica™ Signal System [ESS]). ESS uses two algorithms for routine analyses: empirical Bayes Multi-item Gamma Poisson Shrinker and logistic regression (LR).
  
  
  METHODS
  A model was developed comprising 14 medicines, categorized as "new" or "established." A standard was prepared on the basis of safety findings selected from traditional sources. ESS results were compared with the standard to calculate the positive predictive value (PPV), specificity, and sensitivity. PPVs of the lower one-sided 5% and 0.05% confidence limits of the Bayes geometric mean (EB05) and of the LR odds ratio (LR0005) almost coincided for all the drug-event combinations studied.
  
  
  RESULTS
  There was no obvious difference comparing the PPV of the leading Medical Dictionary for Regulatory Activities (MedDRA) terms to the PPV for all terms. The PPV of narrow MedDRA query searches was higher than that for broad searches. The widely used threshold value of EB05 = 2.0 or LR0005 = 2.0 together with more than three spontaneous reports of the drug-event combination produced balanced results for PPV, sensitivity, and specificity.
  
  
  CONCLUSIONS
  Consequently, performance characteristics were best for leading terms with narrow MedDRA query searches irrespective of applying Multi-item Gamma Poisson Shrinker or LR at a threshold value of 2.0. This research formed the basis for the configuration of ESS for signal detection at Novartis.
authors: Novartis Pharma AG, Basel, Switzerland. conny.berlin@novartis.com
authors: Conny
authors: C
authors: Berlin
authors: Carles
authors: C
authors: Blanch
authors: David J
authors: DJ
authors: Lewis
authors: Dionigi D
authors: DD
authors: Maladorno
authors: Christiane
authors: C
authors: Michel
authors: Michael
authors: M
authors: Petrin
authors: Severine
authors: S
authors: Sarp
authors: Philippe
authors: P
authors: Close
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems / Q000592:standards / Q000706:statistics & numerical data*; D000465:Algorithms; D001499:Bayes Theorem; D003198:Computer Simulation; D057225:Data Mining / Q000592:standards / Q000706:statistics & numerical data*; D064420:Drug-Related Side Effects and Adverse Reactions; D006801:Humans; D016015:Logistic Models; D015233:Models, Statistical; D016012:Poisson Distribution; D011237:Predictive Value of Tests; D011358:Product Surveillance, Postmarketing / Q000592:standards / Q000706:statistics & numerical data*; D012680:Sensitivity and Specificity