title: Conservative Tests under Satisficing Models of Publication Bias.
abstract: Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. We calculate critical values for statistical significance that could help to adjust after the fact for the distortions created by this selection effect, assuming that the only source of publication bias is file drawer bias. These adjusted critical values are easy to calculate and differ from unadjusted critical values by approximately 50%-rather than rejecting a null hypothesis when the t-ratio exceeds 2, the analysis suggests rejecting a null hypothesis when the t-ratio exceeds 3. Samples of published social science research indicate that on average, across research fields, approximately 30% of published t-statistics fall between the standard and adjusted cutoffs.
authors.affiliation: School of Law, University of California, Berkeley, California, United States of America.|National Bureau of Economic Research (NBER), Cambridge, Massachusetts, United States of America.
authors.forename: Justin
authors.initials: J
authors.lastname: McCrary
authors.affiliation: Berkeley Initiative for Transparency in the Social Sciences (BITSS), University of California, Berkeley, California, United States of America.|Berkeley Institute for Data Science (BIDS), University of California, Berkeley, California, United States of America.
authors.forename: Garret
authors.initials: G
authors.lastname: Christensen
authors.affiliation: Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America.
authors.forename: Daniele
authors.initials: D
authors.lastname: Fanelli
mesh_terms: D000465:Algorithms; D006801:Humans; D015233:Models, Statistical; D017594:Publication Bias