Error in statistical tests of error in statistical tests.
BACKGROUND
A recent paper found that terminal digits of statistical values in Nature deviated significantly from an equiprobable distribution, indicating errors or inconsistencies in rounding. This finding, as well as the discovery that a large percentage of p values were inconsistent with reported test statistics, led to a great deal of concern in the popular press and scientific community. The findings ultimately led to new guidelines for all Nature Research Journals.


METHODS
We checked the statistical analysis behind the original paper's tests of equiprobability.


RESULTS
The original paper tested equiprobability with the Kolmogorov-Smirnov test outside its regime of validity. Correct tests find no statistically significant deviations from equiprobability for the statistical values in Nature.


CONCLUSIONS
Statistical tests should be used correctly.
Jeng
Monwhea
M
Department of Physics, Box 1654, Southern Illinois University Edwardsville, Edwardsville, IL 62025, USA. mjeng@physics.syr.edu
D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D017408:Guidelines as Topic; D006801:Humans; D010506:Periodicals as Topic / Q000592:standards* / Q000706:statistics & numerical data; D011786:Quality Control; D012107:Research Design / Q000706:statistics & numerical data*; D016008:Statistical Distributions