title: Meta-research: The art of getting it wrong.
abstract: Meta-analysis has major strengths, but sometimes it can often lead to wrong and misleading answers. In this SRSM presidential address, I discuss some case studies that exemplify these problems, including examples from meta-analyses of both clinical trials and observational associations. I also discuss issues of effect size estimation, bias (in particular significance-chasing biases), and credibility in meta-research. I examine the factors that affect the credibility of meta-analyses, including magnitude of effects, multiplicity of analyses, scale of data, flexibility of analyses, reporting, and conflicts of interest. Under the current circumstances, a survey of expert meta-analysts attending the SRSM meeting showed that most of them believe that the true effect is practically equally likely to lie within the 95% confidence interval of a meta-analysis or outside of it. Finally, I address the placement of meta-analysis in the wider current research agenda and make a plea for adoption of more prospective meta-designs. In many/most/all fields, all primary original research may be designed, executed, and interpreted as a prospective meta-analysis. Copyright © 2011 John Wiley & Sons, Ltd.
authors.affiliation: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. jioannid@stanford.edu, jioannid@cc.uoi.gr.|Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, U.S.A.. jioannid@stanford.edu, jioannid@cc.uoi.gr.|Department of Medicine, Tufts University School of Medicine, Boston, U.S.A.. jioannid@stanford.edu, jioannid@cc.uoi.gr.|Center for Genetic Epidemiology and Modeling, ICRHPS, Tufts Medical Center, Boston, U.S.A.. jioannid@stanford.edu, jioannid@cc.uoi.gr.|Genetics/Genomics, Tufts Clinical and Translational Science Institute, Botson, U.S.A.. jioannid@stanford.edu, jioannid@cc.uoi.gr.|Department of Epidemiology, Harvard School of Public Health, Boston, U.S.A.. jioannid@stanford.edu, jioannid@cc.uoi.gr.
authors.forename: John P A
authors.initials: JP
authors.lastname: Ioannidis
keywords: bias; effect size; meta‐analysis; reporting bias