title: On the reproducibility of meta-analyses: six practical recommendations.
abstract: BACKGROUND
  Meta-analyses play an important role in cumulative science by combining information across multiple studies and attempting to provide effect size estimates corrected for publication bias. Research on the reproducibility of meta-analyses reveals that errors are common, and the percentage of effect size calculations that cannot be reproduced is much higher than is desirable. Furthermore, the flexibility in inclusion criteria when performing a meta-analysis, combined with the many conflicting conclusions drawn by meta-analyses of the same set of studies performed by different researchers, has led some people to doubt whether meta-analyses can provide objective conclusions.
  
  
  CONCLUSIONS
  The present article highlights the need to improve the reproducibility of meta-analyses to facilitate the identification of errors, allow researchers to examine the impact of subjective choices such as inclusion criteria, and update the meta-analysis after several years. Reproducibility can be improved by applying standardized reporting guidelines and sharing all meta-analytic data underlying the meta-analysis, including quotes from articles to specify how effect sizes were calculated. Pre-registration of the research protocol (which can be peer-reviewed using novel 'registered report' formats) can be used to distinguish a-priori analysis plans from data-driven choices, and reduce the amount of criticism after the results are known. The recommendations put forward in this article aim to improve the reproducibility of meta-analyses. In addition, they have the benefit of "future-proofing" meta-analyses by allowing the shared data to be re-analyzed as new theoretical viewpoints emerge or as novel statistical techniques are developed. Adoption of these practices will lead to increased credibility of meta-analytic conclusions, and facilitate cumulative scientific knowledge.
authors.affiliation: Human Technology Interaction Group, Eindhoven University of Technology, IPO 1.33, PO Box 513, 5600MB, Eindhoven, The Netherlands. Lakens@gmail.com.
authors.forename: Daniël
authors.initials: D
authors.lastname: Lakens
authors.affiliation: University of Missouri, Missouri, USA.
authors.forename: Joe
authors.initials: J
authors.lastname: Hilgard
authors.affiliation: University of Amsterdam, Amsterdam, The Netherlands.
authors.forename: Janneke
authors.initials: J
authors.lastname: Staaks
mesh_terms: D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias; D015203:Reproducibility of Results
keywords: Meta-analysis; Open science; Reporting guidelines; Reproducibility