title: Comparing the accuracy of experimental estimates to guessing: a new perspective on replication and the "Crisis of Confidence" in psychology.
abstract: We develop a general measure of estimation accuracy for fundamental research designs, called v. The v measure compares the estimation accuracy of the ubiquitous ordinary least squares (OLS) estimator, which includes sample means as a special case, with a benchmark estimator that randomizes the direction of treatment effects. For sample and effect sizes common to experimental psychology, v suggests that OLS produces estimates that are insufficiently accurate for the type of hypotheses being tested. We demonstrate how v can be used to determine sample sizes to obtain minimum acceptable estimation accuracy. Software for calculating v is included as online supplemental material (R Core Team, 2012).
authors: Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA, stoberc@missouri.edu.
authors: Clintin P
authors: CP
authors: Davis-Stober
authors: Jason
authors: J
authors: Dana
mesh_terms: D000704:Analysis of Variance; D019985:Benchmarking / Q000379:methods; D003627:Data Interpretation, Statistical; D006801:Humans; D016018:Least-Squares Analysis; D015201:Meta-Analysis as Topic; D011589:Psychology, Experimental / Q000379:methods* / Q000592:standards*; D012044:Regression Analysis; D015203:Reproducibility of Results; D012107:Research Design; D018401:Sample Size; D012984:Software; D012985:Software Design