title: Using Nonnaive Participants Can Reduce Effect Sizes.
abstract: Although researchers often assume their participants are naive to experimental materials, this is not always the case. We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in the first. The reduction was most pronounced when participants were assigned to a different condition in the second wave. We discuss the methodological implications of these findings.
authors.affiliation: Institute for Social Research, University of Michigan Mathematica Policy Research, Ann Arbor, MI jjchandl@umich.edu.
authors.forename: Jesse
authors.initials: J
authors.lastname: Chandler
authors.affiliation: Rotterdam School of Management, Erasmus University Rotterdam.
authors.forename: Gabriele
authors.initials: G
authors.lastname: Paolacci
authors.affiliation: Graduate School of Business Administration, Bar-Ilan University.
authors.forename: Eyal
authors.initials: E
authors.lastname: Peer
authors.affiliation: Department of Psychology, Princeton University.
authors.forename: Pam
authors.initials: P
authors.lastname: Mueller
authors.affiliation: Department of Psychology, University of Florida.
authors.forename: Kate A
authors.initials: KA
authors.lastname: Ratliff
mesh_terms: D000328:Adult; D005260:Female; D006801:Humans; D008297:Male; D010358:Patient Participation / Q000379:methods*; D018579:Patient Selection
keywords: effect sizes; judgment and decision making; nonnaïveté; open data; open materials; panel conditioning; repeated participation; research methods