title: The Scientific Impact of Positive and Negative Phase 3 Cancer Clinical Trials.
abstract: OBJECTIVE
  Positive phase 3 cancer clinical trials are widely hailed, while trials with negative results are often interpreted as scientific failures. We hypothesized that these interpretations would be reflected in the scientific literature.
  
  
  OBJECTIVE
  To compare the scientific impact of positive vs negative phase 3 cancer clinical treatment trials.
  
  
  METHODS
  We examined the phase 3 trial history of SWOG, a national cancer clinical trials consortium, over a 30-year period (1985-2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar. Citation counts were compared using generalized estimating equations for Poisson regression. Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Trials were categorized as positive if they achieved a statistically significant result in favor of the new experimental treatment for the protocol-specified primary end point. Trials were categorized as negative if they achieved a statistically significant result in favor of standard therapy or a null result with no statistically significant benefit for either the experimental or standard therapy.
  
  
  METHODS
  Impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial articles and all secondary articles associated with the trials.
  
  
  RESULTS
  Ninety-four studies enrolling n = 46 424 patients were analyzed. Twenty-eight percent of trials were positive (26 of 94). The primary publications from positive trials were published in journals with higher mean (SD) 2-year impact factors (28 [19] vs 18 [13]; P = .007) and were cited twice as often as negative trials (mean per year, 43 vs 21; relative risk, 2.0; 95% CI, 1.1-3.9; P = .03). However, the number of citations from all primary and secondary articles did not significantly differ between positive and negative trials (mean per year, 55 vs 45; relative risk, 1.2; 95% CI, 0.7-2.3; P = .53).
  
  
  CONCLUSIONS
  The scientific impact of the primary articles from positive phase 3 randomized cancer clinical trials was twice as great as for negative trials. But when all of the articles associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.
authors.affiliation: SWOG Statistical Center, Fred Hutchinson Cancer Research Center, Seattle, Washington.
authors.forename: Joseph M
authors.initials: JM
authors.lastname: Unger
authors.affiliation: SWOG Statistical Center, Fred Hutchinson Cancer Research Center, Seattle, Washington.
authors.forename: William E
authors.initials: WE
authors.lastname: Barlow
authors.affiliation: Fred Hutchinson Cancer Research Center, Seattle, Washington.
authors.forename: Scott D
authors.initials: SD
authors.lastname: Ramsey
authors.affiliation: SWOG Statistical Center, Fred Hutchinson Cancer Research Center, Seattle, Washington.
authors.forename: Michael
authors.initials: M
authors.lastname: LeBlanc
authors.affiliation: SWOG Group Chair's Office/Knight Cancer Institute, Oregon Health and Science University, Portland.
authors.forename: Charles D
authors.initials: CD
authors.lastname: Blanke
authors.affiliation: Columbia University, New York, New York.
authors.forename: Dawn L
authors.initials: DL
authors.lastname: Hershman
mesh_terms: D017326:Clinical Trials, Phase III as Topic / Q000706:statistics & numerical data*; D006801:Humans; D055811:Journal Impact Factor; D008495:Medical Oncology / Q000706:statistics & numerical data*; D009369:Neoplasms / Q000188:drug therapy / Q000453:epidemiology* / Q000473:pathology; D011643:Publishing