title: The distribution of P-values in medical research articles suggested selective reporting associated with statistical significance.
abstract: OBJECTIVE
  Published P-values provide a window into the global enterprise of medical research. The aim of this study was to use the distribution of published P-values to estimate the relative frequencies of null and alternative hypotheses and to seek irregularities suggestive of publication bias.
  
  
  METHODS
  This cross-sectional study included P-values published in 120 medical research articles in 2016 (30 each from the BMJ, JAMA, Lancet, and New England Journal of Medicine). The observed distribution of P-values was compared with expected distributions under the null hypothesis (i.e., uniform between 0 and 1) and the alternative hypothesis (strictly decreasing from 0 to 1). P-values were categorized according to conventional levels of statistical significance and in one-percent intervals.
  
  
  RESULTS
  Among 4,158 recorded P-values, 26.1% were highly significant (P < 0.001), 9.1% were moderately significant (P ≥ 0.001 to < 0.01), 11.7% were weakly significant (P ≥ 0.01 to < 0.05), and 53.2% were nonsignificant (P ≥ 0.05). We noted three irregularities: (1) high proportion of P-values <0.001, especially in observational studies, (2) excess of P-values equal to 1, and (3) about twice as many P-values less than 0.05 compared with those more than 0.05. The latter finding was seen in both randomized trials and observational studies, and in most types of analyses, excepting heterogeneity tests and interaction tests. Under plausible assumptions, we estimate that about half of the tested hypotheses were null and the other half were alternative.
  
  
  CONCLUSIONS
  This analysis suggests that statistical tests published in medical journals are not a random sample of null and alternative hypotheses but that selective reporting is prevalent. In particular, significant results are about twice as likely to be reported as nonsignificant results.
authors.affiliation: Division of Clinical Epidemiology, Faculty of Medicine, University of Geneva, and Geneva University Hospitals, 6 rue Gabrielle-Perret-Gentil, CH-1211 Geneva, Switzerland. Electronic address: thomas.perneger@hcuge.ch.
authors.forename: Thomas V
authors.initials: TV
authors.lastname: Perneger
authors.affiliation: Division of Clinical Epidemiology, Faculty of Medicine, University of Geneva, and Geneva University Hospitals, 6 rue Gabrielle-Perret-Gentil, CH-1211 Geneva, Switzerland.
authors.forename: Christophe
authors.initials: C
authors.lastname: Combescure
mesh_terms: D035843:Biomedical Research / Q000379:methods* / Q000706:statistics & numerical data*; D003430:Cross-Sectional Studies; D006801:Humans; D010506:Periodicals as Topic / Q000706:statistics & numerical data*; D017594:Publication Bias / Q000706:statistics & numerical data*; D013223:Statistics as Topic / Q000379:methods*
keywords: P-values; Practice of research; Publication bias; Statistical tests