title: Heterogeneity in ecological and evolutionary meta-analyses: its magnitude and implications.
abstract: Meta-analysis is the gold standard for synthesis in ecology and evolution. Together with estimating overall effect magnitudes, meta-analyses estimate differences between effect sizes via heterogeneity statistics. It is widely hypothesized that heterogeneity will be present in ecological/evolutionary meta-analyses due to the system-specific nature of biological phenomena. Despite driving recommended best practices, the generality of heterogeneity in ecological data has never been systematically reviewed. We reviewed 700 studies, finding 325 that used formal meta-analysis, of which total heterogeneity was reported in fewer than 40%. We used second-order meta-analysis to collate heterogeneity statistics from 86 studies. Our analysis revealed that the median and mean heterogeneity, expressed as I2, are 84.67% and 91.69%, respectively. These estimates are well above "high" heterogeneity (i.e., 75%), based on widely adopted benchmarks. We encourage reporting heterogeneity in the forms of I2and the estimated variance components (e.g., τ2) as standard practice. These statistics provide vital insights in to the degree to which effect sizes vary, and provide the statistical support for the exploration of predictors of effect-size magnitude. Along with standard meta-regression techniques that fit moderator variables, multi-level models now allow partitioning of heterogeneity among correlated (e.g., phylogenetic) structures that exist within data.
authors.affiliation: Charles Perkins Centre, The University of Sydney, Sydney, New South Wales, Australia.|School of Mathematics and Statistics, The University of Sydney, Sydney, New South Wales, Australia.
authors.forename: Alistair M
authors.initials: AM
authors.lastname: Senior
authors.affiliation: School of Life and Environmental Sciences, Faculty of Veterinary Science, The University of Sydney, Sydney, New South Wales, Australia.|San Diego Zoo Global, San Diego, California, 92112, USA.
authors.forename: Catherine E
authors.initials: CE
authors.lastname: Grueber
authors.affiliation: Department of Ecology and Evolutionary Biology, University of Toronto, Toronto, Ontario, M5S 3B2, Canada.
authors.forename: Tsukushi
authors.initials: T
authors.lastname: Kamiya
authors.affiliation: Evolution and Ecology Research Centre, and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, New South Wales, Australia.
authors.forename: Malgorzata
authors.initials: M
authors.lastname: Lagisz
authors.affiliation: Department of Chemistry and Biology, Ryerson University, Toronto, Ontario, M5B 2K3, Canada.
authors.forename: Katie
authors.initials: K
authors.lastname: O'Dwyer
authors.affiliation: BECO do Departamento de Zoologia, Instituto de Biociências, Universidade de São Paulo, Rua do Matão, trav. 14, no. 321, São Paulo, São Paulo, 05508-090, Brazil.
authors.forename: Eduardo S A
authors.initials: ES
authors.lastname: Santos
authors.affiliation: Evolution and Ecology Research Centre, and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, New South Wales, Australia.
authors.forename: Shinichi
authors.initials: S
authors.lastname: Nakagawa
mesh_terms: D015982:Bias; D005075:Biological Evolution; D003627:Data Interpretation, Statistical; D004463:Ecology / Q000379:methods*; D015201:Meta-Analysis as Topic; D015233:Models, Statistical; D012107:Research Design
keywords: 
  I
  2
  ; Cochran's Q; eco-evolutionary meta-analysis; effect size; homogeneity; meta-regression; mixed model; phylogenetic signal/heritability; quantitative review; sampling variance; systematic review; weighted regression