Statistical conclusion validity: an empirical analysis of multiplicity in mental retardation research.
The statistical conclusion validity of mental retardation research was evaluated by reviewing 41 recently published articles in the American Journal on Mental Retardation and the Journal of Mental Deficiency Research. The analysis was focused on the incidence of Type I error as reflected by three different measures: the experiment-wise error rate, the error rate per experiment, and the percentage error rate. Values for each of these measures indicated that the probability of committing a Type I error was considerably greater than the traditionally assumed level of p less than .05, or 5%. The percentage error rate computed for the 41 articles (369 statistical tests) suggested that approximately 20% of the statistically significant results may be erroneous. Procedures to evaluate the impact of Type I errors in mental retardation research were presented and briefly discussed.
Ottenbacher
K J
KJ
SUNY, Buffalo 14214.
D000203:Activities of Daily Living / Q000523:psychology; D015982:Bias; D003627:Data Interpretation, Statistical; D004495:Education of Persons with Intellectual Disabilities / Q000706:statistics & numerical data*; D005500:Follow-Up Studies; D006801:Humans; D008607:Intellectual Disability / Q000534:rehabilitation*; D015233:Models, Statistical