title: Low power, type II errors, and other statistical problems in recent cardiovascular research.
abstract: Frequently in biomedical literature, measurements are considered "not statistically different" if a statistical test fails to achieve a P value that is < or = 0.05. This conclusion may be misleading because the size of each group is too small or the variability is large, and a type II error (false negative) is committed. In this study, we examined the probabilities of detecting a real difference (power) and type II errors in unpaired t-tests in Volumes 246 and 266 of the American Journal of Physiology: Heart and Circulatory Physiology. In addition, we examined all articles for other statistical errors. The median power of the t-tests was similar in these volumes (approximately 0.55 and approximately 0.92 to detect a 20% and a 50% change, respectively). In both volumes, approximately 80% of the studies with nonsignificant unpaired t-tests contained at least one t-test with a type II error probability > 0.30. Our findings suggest that low power and a high incidence of type II errors are common problems in this journal. In addition, the presentation of statistics was often vague, t-tests were misused frequently, and assumptions for inferential statistics usually were not mentioned or examined.
authors.affiliation: Department of Physiology and Pharmacology, School of Medicine, University of South Dakota, Vermillion 57069, USA.
authors.forename: J L
authors.initials: JL
authors.lastname: Williams
authors.forename: C A
authors.initials: CA
authors.lastname: Hathaway
authors.forename: K L
authors.initials: KL
authors.lastname: Kloster
authors.forename: B H
authors.initials: BH
authors.lastname: Layne
mesh_terms: D000704:Analysis of Variance; D002319:Cardiovascular System; D005163:Factor Analysis, Statistical; D010506:Periodicals as Topic; D010827:Physiology; D012044:Regression Analysis; D015203:Reproducibility of Results; D012106:Research / Q000592:standards*; D012107:Research Design; D013223:Statistics as Topic; D018709:Statistics, Nonparametric; D014481:United States