title: Meta-analysis to integrate effect sizes within an article: Possible misuse and Type I error inflation.
abstract: In recent years an increasing number of articles have employed meta-analysis to integrate effect sizes of researchers' own series of studies within a single article ("internal meta-analysis"). Although this approach has the obvious advantage of obtaining narrower confidence intervals, we show that it could inadvertently inflate false-positive rates if researchers are motivated to use internal meta-analysis in order to obtain a significant overall effect. Specifically, if one decides whether to stop or continue a further replication experiment depending on the significance of the results in an internal meta-analysis, false-positive rates would increase beyond the nominal level. We conducted a set of Monte Carlo simulations to demonstrate our argument, and provided a literature review to gauge awareness and prevalence of this issue. Furthermore, we made several recommendations when using internal meta-analysis to make a judgment on statistical significance.
authors.affiliation: Department of Psychology, University of Reading.
authors.forename: Taiji
authors.initials: T
authors.lastname: Ueno
authors.affiliation: Department of Psychology, University of Reading.
authors.forename: Greta M
authors.initials: GM
authors.lastname: Fastrich
authors.affiliation: Department of Psychology, University of Reading.
authors.forename: Kou
authors.initials: K
authors.lastname: Murayama
mesh_terms: D006801:Humans; D015201:Meta-Analysis as Topic; D009010:Monte Carlo Method; D010506:Periodicals as Topic; D011584:Psychology