title: Agreements and Discrepancies between FDA Reports and Journal Papers on Biologic Agents Approved for Rheumatoid Arthritis: A Meta-Research Project.
abstract: BACKGROUND
  Sponsors that seek to commercialize new drugs apply to the Food and Drug Administration (FDA) which independently analyzes the raw data and reports the results on its website.
  
  
  OBJECTIVE
  This study sought to determine if there are differences between the FDA assessments and journal reports on biologic agents developed for the treatment of rheumatoid arthritis.
  
  
  METHODS
  Available data on FDA-approved drugs were extracted from the website, and a systematic literature search was conducted to identify matching studies in peer-reviewed medical journals. Outcome measures were the American College of Rheumatology response criteria ACR20 (efficacy) and withdrawal due to adverse events (safety). As effect size odds ratios were estimated for each active trial arm vs. control arm (i.e. for both sources: FDA and journal report), followed by calculation of the ratios of the FDA and journal report odds ratios. A ratio of odds ratios not equal to 1 was categorized as a discrepancy.
  
  
  RESULTS
  FDA reports were available for 8 of 9 FDA-approved biologic agents for rheumatoid arthritis; all identified trials (34) except one were published in peer-reviewed journals. Overall, discrepancies were noted for 20 of the 33 evaluated trials. Differences in the apparent benefit reporting were found in 39% (24/61) pairwise comparisons and in 11 cases these were statistically significant; the FDA report showed greater benefit than the journal publication in 15 comparisons and lesser benefit in 9. Differences in the reported harms were found in 51% (28/55) pairwise comparisons and were statistically significant in 5. The "signal" in FDA reports showed a less harmful effect than the journal publication in 17 comparisons whereas a more harmful effect in 11. The differences were attributed to differences in analytic approach, patient inclusion, rounding effect, and counting discrepancies. However, no differences were categorized as critical.
  
  
  CONCLUSIONS
  There was no empirical evidence to suggest biased estimates between the two sources. Increased and detailed transparency in publications would improve the understanding and credibility of published results. Further, the FDA report was found to be a useful source when data are missing in the published report (i.e. reporting bias).
authors.affiliation: Pediatric Rheumatology Unit, Schneider Children's Medical Center of Israel, Petach Tikva, and Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
authors.forename: Gil
authors.initials: G
authors.lastname: Amarilyo
authors.affiliation: David Geffen School of Medicine, University of California, Los Angeles, California, United States of America.
authors.forename: Daniel E
authors.initials: DE
authors.lastname: Furst
authors.affiliation: David Geffen School of Medicine, University of California, Los Angeles, California, United States of America.
authors.forename: Jennifer M P
authors.initials: JM
authors.lastname: Woo
authors.affiliation: David Geffen School of Medicine, University of California, Los Angeles, California, United States of America.
authors.forename: Wen
authors.initials: W
authors.lastname: Li
authors.affiliation: Musculoskeletal Statistics Unit, The Parker Institute, Department of Rheumatology, Copenhagen University Hospital, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark.
authors.forename: Henning
authors.initials: H
authors.lastname: Bliddal
authors.affiliation: Musculoskeletal Statistics Unit, The Parker Institute, Department of Rheumatology, Copenhagen University Hospital, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark.
authors.forename: Robin
authors.initials: R
authors.lastname: Christensen
authors.affiliation: Musculoskeletal Statistics Unit, The Parker Institute, Department of Rheumatology, Copenhagen University Hospital, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark.
authors.forename: Simon
authors.initials: S
authors.lastname: Tarp
mesh_terms: D001172:Arthritis, Rheumatoid / Q000628:therapy*; D001688:Biological Products / Q000627:therapeutic use*; D017277:Drug Approval / Q000331:legislation & jurisprudence*; D006801:Humans; D016032:Randomized Controlled Trials as Topic; D014481:United States; D014486:United States Food and Drug Administration
chemical_list: D001688:Biological Products