The consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Although the consequences of ignoring a nested factor on decisions to reject the null hypothesis of no treatment effects have been discussed in the literature, typically researchers in applied psychology and education ignore treatment providers (often a nested factor) when comparing the efficacy of treatments. The incorrect analysis, however, not only invalidates tests of hypotheses, but it also overestimates the treatment effect. Formulas were derived and a Monte Carlo study was conducted to estimate the degree to which the F statistic and treatment effect size measures are inflated by ignoring the effects due to providers of treatments. These untoward effects are illustrated with examples from psychotherapeutic treatments.
Wampold
B E
BE
Department of Counseling Psychology, 321 Education Building, 1000 Bascom Mall, University of Wisconsin-Madison, Madison, Wisconsin 53706, USA. wampold@education.wisc.edu
Serlin
R C
RC
D000704:Analysis of Variance; D006801:Humans; D015233:Models, Statistical; D010043:Outcome and Process Assessment, Health Care / Q000706:statistics & numerical data*; D011594:Psychometrics; D011613:Psychotherapy / Q000706:statistics & numerical data*