title: Early detection of pharmacovigilance signals with automated methods based on false discovery rates: a comparative study.
abstract: BACKGROUND
  Improving the detection of drug safety signals has led several pharmacovigilance regulatory agencies to incorporate automated quantitative methods into their spontaneous reporting management systems. The three largest worldwide pharmacovigilance databases are routinely screened by the lower bound of the 95% confidence interval of proportional reporting ratio (PRR₀₂.₅), the 2.5% quantile of the Information Component (IC₀₂.₅) or the 5% quantile of the Gamma Poisson Shrinker (GPS₀₅). More recently, Bayesian and non-Bayesian False Discovery Rate (FDR)-based methods were proposed that address the arbitrariness of thresholds and allow for a built-in estimate of the FDR. These methods were also shown through simulation studies to be interesting alternatives to the currently used methods.
  
  
  OBJECTIVE
  The objective of this work was twofold. Based on an extensive retrospective study, we compared PRR₀₂.₅, GPS₀₅ and IC₀₂.₅ with two FDR-based methods derived from the Fisher's exact test and the GPS model (GPS(pH0) [posterior probability of the null hypothesis H₀ calculated from the Gamma Poisson Shrinker model]). Secondly, restricting the analysis to GPS(pH0), we aimed to evaluate the added value of using automated signal detection tools compared with 'traditional' methods, i.e. non-automated surveillance operated by pharmacovigilance experts.
  
  
  METHODS
  The analysis was performed sequentially, i.e. every month, and retrospectively on the whole French pharmacovigilance database over the period 1 January 1996-1 July 2002. Evaluation was based on a list of 243 reference signals (RSs) corresponding to investigations launched by the French Pharmacovigilance Technical Committee (PhVTC) during the same period. The comparison of detection methods was made on the basis of the number of RSs detected as well as the time to detection.
  
  
  RESULTS
  Results comparing the five automated quantitative methods were in favour of GPS(pH0) in terms of both number of detections of true signals and time to detection. Additionally, based on an FDR threshold of 5%, GPS(pH0) detected 87% of the RSs associated with more than three reports, anticipating the date of investigation by the PhVTC by 15.8 months on average.
  
  
  CONCLUSIONS
  Our results show that as soon as there is reasonable support for the data, automated signal detection tools are powerful tools to explore large spontaneous reporting system databases and detect relevant signals quickly compared with traditional pharmacovigilance methods.
authors.affiliation: INSERM, CESP Centre for Research in Epidemiology and Population Health, U1018, Biostatistics, Villejuif, France. ismail.ahmed@inserm.fr
authors.forename: Ismaïl
authors.initials: I
authors.lastname: Ahmed
authors.forename: Frantz
authors.initials: F
authors.lastname: Thiessard
authors.forename: Ghada
authors.initials: G
authors.lastname: Miremont-Salamé
authors.forename: Françoise
authors.initials: F
authors.lastname: Haramburu
authors.forename: Carmen
authors.initials: C
authors.lastname: Kreft-Jais
authors.forename: Bernard
authors.initials: B
authors.lastname: Bégaud
authors.forename: Pascale
authors.initials: P
authors.lastname: Tubert-Bitter
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems / Q000592:standards*; D016208:Databases, Factual; D064420:Drug-Related Side Effects and Adverse Reactions; D005189:False Positive Reactions; D005602:France; D006801:Humans; D015233:Models, Statistical; D010363:Pattern Recognition, Automated / Q000379:methods*; D060735:Pharmacovigilance; D012189:Retrospective Studies; D013997:Time Factors