Accuracy when inferential statistics are used as measurement tools.
BACKGROUND
Inferential statistical tests that approximate measurement are called acceptance procedures. The procedure includes type 1 error, falsely rejecting the null hypothesis, and type 2 error, failing to reject the null hypothesis when the alternative should be supported. This approach involves repeated sampling from a distribution with established parameters such that the probabilities of these errors can be ascertained. With low error probabilities the procedure has the potential to approximate measurement. How close this procedure approximates measurement was examined.


RESULTS
A Monte Carlo procedure set the type 1 error at p = 0.05 and the type 2 error at either p = 0.20 or p = 0.10 for effect size values of d = 0.2, 0.5, and 0.8. The resultant values are approximately 15 and 6.25% larger than the effect sizes entered into the analysis depending on a type 2 error rate of p < 0.20, or p < 0.10 respectively.


CONCLUSIONS
Acceptance procedures approximate values wherein a decision could be made. In a health district a deviation at a particular level could signal a change in health. The approximations could be reasonable in some circumstances, but if more accurate measures are desired a deviation could be reduced by the percentage appropriate for the power. The tradeoff for such a procedure is an increase in type 1 error rate and a decrease in type 2 errors.
Department of Psychology, University of New Brunswick, 100 Tucker Park Road, PO Box 5050, Saint John, NB, E2L 4L5, Canada. Bradley@unb.ca.
Michael T
MT
Bradley
NWORTH Bangor Clinical Trials Unit, Institute of Medical & Social Care Research, Bangor University, Bangor, UK.
Andrew
A
Brand
D015233:Models, Statistical; D009010:Monte Carlo Method; D011336:Probability; D015203:Reproducibility of Results; D012586:Science / Q000379:methods / Q000706:statistics & numerical data; D013223:Statistics as Topic / Q000379:methods*
Accuracy; Measurement; Neyman and Pearson; Probabilities; Type 1 errors; Type 2 errors