Analysis of categorical moderators in mixed-effects meta-analysis: Consequences of using pooled versus separate estimates of the residual between-studies variances.
Subgroup analyses allow us to examine the influence of a categorical moderator on the effect size in meta-analysis. We conducted a simulation study using a dichotomous moderator, and compared the impact of pooled versus separate estimates of the residual between-studies variance on the statistical performance of the QB(P)and QB(S)tests for subgroup analyses assuming a mixed-effects model. Our results suggested that similar performance can be expected as long as there are at least 20 studies and these are approximately balanced across categories. Conversely, when subgroups were unbalanced, the practical consequences of having heterogeneous residual between-studies variances were more evident, with both tests leading to the wrong statistical conclusion more often than in the conditions with balanced subgroups. A pooled estimate should be preferred for most scenarios, unless the residual between-studies variances are clearly different and there are enough studies in each category to obtain precise separate estimates.
Department of Basic Psychology & Methodology, Faculty of Psychology, University of Murcia, Spain.
María
M
Rubio-Aparicio
Department of Basic Psychology & Methodology, Faculty of Psychology, University of Murcia, Spain.
Julio
J
Sánchez-Meca
School of Social and Community Medicine, University of Bristol, UK.
José Antonio
JA
López-López
Department of Social Psychology & Methodology, Faculty of Psychology, Autonomous University of Madrid, Spain.
Juan
J
Botella
Department of Basic Psychology & Methodology, Faculty of Psychology, University of Murcia, Spain.
Fulgencio
F
Marín-Martínez
D006801:Humans; D000704:Analysis of Variance; D056808:Biostatistics / Q000379:methods; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D015233:Models, Statistical; D011584:Psychology / Q000706:statistics & numerical data; D015201:Meta-Analysis as Topic
between-studies variance; meta-analysis; mixed-effects model; subgroup analysis