title: Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording.
abstract: Observer bias and other "experimenter effects" occur when researchers' expectations influence study outcome. These biases are strongest when researchers expect a particular result, are measuring subjective variables, and have an incentive to produce data that confirm predictions. To minimize bias, it is good practice to work "blind," meaning that experimenters are unaware of the identity or treatment group of their subjects while conducting research. Here, using text mining and a literature review, we find evidence that blind protocols are uncommon in the life sciences and that nonblind studies tend to report higher effect sizes and more significant p-values. We discuss methods to minimize bias and urge researchers, editors, and peer reviewers to keep blind protocols in mind.
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Canberra, Australian Capital Territory, Australia.
authors.forename: Luke
authors.initials: L
authors.lastname: Holman
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Canberra, Australian Capital Territory, Australia.
authors.forename: Megan L
authors.initials: ML
authors.lastname: Head
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Canberra, Australian Capital Territory, Australia; Department of Biological Sciences, Macquarie University, Sydney, New South Wales, Australia.
authors.forename: Robert
authors.initials: R
authors.lastname: Lanfear
authors.affiliation: Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Canberra, Australian Capital Territory, Australia.
authors.forename: Michael D
authors.initials: MD
authors.lastname: Jennions
mesh_terms: D001695:Biology / Q000592:standards* / Q000706:statistics & numerical data; D003625:Data Collection / Q000592:standards* / Q000706:statistics & numerical data; D057225:Data Mining