Treatment outcomes for students with learning disabilities: how important are internal and external validity?
This study analyzed the magnitude of experimental intervention outcomes as a function of violations in internal and external validity for studies that included students with learning disabilities. The results indicated that treatment outcomes were significantly affected by the following violations: teacher effects, establishing criterion levels of instructional performance, reliance on experimental measures, using different measures between pretest and posttest, using a sample heterogenous in age, and using incorrect units of analysis. Furthermore, the underreporting of information related to ethnicity, locale of the study, psychometric data, and teacher applications positively inflated the magnitude of treatment outcomes. A weighted hierarchical regression analysis revealed that composite scores of the aforementioned high-risk variables accounted for 16% of the total variance in effect size. The implications for interpreting intervention research to practice are discussed.
Simmerman
S
S
Educational Psychology and Special Education, School of Education, University of California, Riverside 92521, USA.
Swanson
H L
HL
D000704:Analysis of Variance; D015982:Bias; D002648:Child; D002986:Clinical Trials as Topic; D004519:Education, Special / Q000379:methods*; D004521:Educational Measurement / Q000706:statistics & numerical data*; D004522:Educational Status; D006801:Humans; D007859:Learning Disabilities / Q000628:therapy*; D017063:Outcome Assessment, Health Care / Q000706:statistics & numerical data*; D015203:Reproducibility of Results