Correcting overestimated effect size estimates in multiple trials.
In a simulation study, Brand, Bradley, Best, and Stoica (2011) have shown that Cohen's d is notably overestimated if computed for data aggregated over multiple trials. Although the phenomenon is highly important for studies and meta-analyses of studies structurally similar to the simulated scenario, the authors do not comprehensively address how the problem could be handled. In this comment, we first suggest a corrective term d(')c that includes the number and correlation of trials. Next, the results of a simulation study provide evidence that the proposed dc' results in a more precise estimation of trial-level effects. We conclude that, in practice, d( ')c together with plausible estimates of inter-trial correlation will produce a more precise effect size range compared to that suggested by Brand and colleagues (2011).
a University of Klagenfurt.
Wolfgang
W
Wiedermann
Bartosz
B
Gula
Paul
P
Czech
Denise
D
Muschik
D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D011589:Psychology, Experimental / Q000379:methods; D012107:Research Design; D018401:Sample Size
Cohen's d effect size; aggregation bias; multiple trials