title: The significance fallacy in inferential statistics.
abstract: BACKGROUND
  Statistical significance is an important concept in empirical science. However the meaning of the term varies widely. We investigate into the intuitive understanding of the notion of significance.
  
  
  METHODS
  We described the results of two different experiments published in a major psychological journal to a sample of students of psychology, labeling the findings as 'significant' versus 'non-significant.' Participants were asked to estimate the effect sizes and sample sizes of the original studies.
  
  
  RESULTS
  Labeling the results of a study as significant was associated with estimations of a big effect, but was largely unrelated to sample size. Similarly, non-significant results were estimated as near zero in effect size.
  
  
  CONCLUSIONS
  After considerable training in statistics, students largely equate statistical significance with medium to large effect sizes, rather than with large sample sizes. The data show that students assume that statistical significance is due to real effects, rather than to 'statistical tricks' (e.g., increasing sample size).
authors.affiliation: Department of Psychology and Centre of Cognitive Neuroscience, University of Salzburg, Hellbrunnerstr. 34, 5020, Salzburg, Austria. Anton.kuehberger@sbg.ac.at.
authors.forename: Anton
authors.initials: A
authors.lastname: Kühberger
authors.affiliation: Österreichisches Zentrum für Begabtenförderung und Begabungsforschung, Salzburg, Austria. Astrid.fritz@begabtenzentrum.at.
authors.forename: Astrid
authors.initials: A
authors.lastname: Fritz
authors.affiliation: Department of Psychology, University of Munich, Munich, Germany. Eva.lermer@psy.lmu.de.
authors.forename: Eva
authors.initials: E
authors.lastname: Lermer
authors.affiliation: Department of Psychology and Centre of Cognitive Neuroscience, University of Salzburg, Hellbrunnerstr. 34, 5020, Salzburg, Austria. Thomas.scherndl@sbg.ac.at.
authors.forename: Thomas
authors.initials: T
authors.lastname: Scherndl
mesh_terms: D000293:Adolescent; D000328:Adult; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008297:Male; D018401:Sample Size; D055815:Young Adult