title: The extent and consequences of p-hacking in science.
abstract: A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as "p-hacking," occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia.
authors.forename: Megan L
authors.initials: ML
authors.lastname: Head
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia.
authors.forename: Luke
authors.initials: L
authors.lastname: Holman
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia; Department of Biological Sciences, Faculty of Science, Macquarie University, North Ryde, New South Wales, Australia.
authors.forename: Rob
authors.initials: R
authors.lastname: Lanfear
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia.
authors.forename: Andrew T
authors.initials: AT
authors.lastname: Kahn
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia.
authors.forename: Michael D
authors.initials: MD
authors.lastname: Jennions
mesh_terms: D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias; D012586:Science / Q000941:ethics* / Q000706:statistics & numerical data; D013223:Statistics as Topic