title: Evaluating replicability of laboratory experiments in economics.
abstract: The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that were made publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs.
authors.affiliation: Division of Humanities and Social Sciences, California Institute of Technology, 1200 East California Boulevard, MC 228-77, Pasadena, CA 91125, USA.
authors.forename: Colin F
authors.initials: CF
authors.lastname: Camerer
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Anna
authors.initials: A
authors.lastname: Dreber
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Eskil
authors.initials: E
authors.lastname: Forsell
authors.affiliation: Haas School of Business, University of California-Berkeley, Berkeley, CA 94720-1900, USA. NUS Business School, National University of Singapore, Singapore 119245.
authors.forename: Teck-Hua
authors.initials: TH
authors.lastname: Ho
authors.affiliation: Department of Banking and Finance, University of Innsbruck, Universitätsstrasse 15, 6020 Innsbruck, Austria.
authors.forename: Jürgen
authors.initials: J
authors.lastname: Huber
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Magnus
authors.initials: M
authors.lastname: Johannesson
authors.affiliation: Department of Banking and Finance, University of Innsbruck, Universitätsstrasse 15, 6020 Innsbruck, Austria. Centre for Finance, Department of Economics, University of Göteborg, SE-40530 Göteborg, Sweden.
authors.forename: Michael
authors.initials: M
authors.lastname: Kirchler
authors.affiliation: Sveriges Riksbank, SE-103 37 Stockholm, Sweden.
authors.forename: Johan
authors.initials: J
authors.lastname: Almenberg
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Adam
authors.initials: A
authors.lastname: Altmejd
authors.affiliation: Office of the Deputy President (Research and Technology), National University of Singapore, Singapore 119077.
authors.forename: Taizan
authors.initials: T
authors.lastname: Chan
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Emma
authors.initials: E
authors.lastname: Heikensten
authors.affiliation: Department of Banking and Finance, University of Innsbruck, Universitätsstrasse 15, 6020 Innsbruck, Austria.
authors.forename: Felix
authors.initials: F
authors.lastname: Holzmeister
authors.affiliation: Division of Humanities and Social Sciences, California Institute of Technology, 1200 East California Boulevard, MC 228-77, Pasadena, CA 91125, USA.
authors.forename: Taisuke
authors.initials: T
authors.lastname: Imai
authors.affiliation: Department of Economics, Stockholm School of Economics, Box 6501, SE-113 83 Stockholm, Sweden.
authors.forename: Siri
authors.initials: S
authors.lastname: Isaksson
authors.affiliation: Division of Humanities and Social Sciences, California Institute of Technology, 1200 East California Boulevard, MC 228-77, Pasadena, CA 91125, USA.
authors.forename: Gideon
authors.initials: G
authors.lastname: Nave
authors.affiliation: New Zealand Institute for Advanced Study, Private Bag 102904, North Shore Mail Centre, Auckland 0745, New Zealand. Wissenschaftskolleg zu Berlin, Institute for Advanced Study, D-14193 Berlin, Germany.
authors.forename: Thomas
authors.initials: T
authors.lastname: Pfeiffer
authors.affiliation: Department of Banking and Finance, University of Innsbruck, Universitätsstrasse 15, 6020 Innsbruck, Austria.
authors.forename: Michael
authors.initials: M
authors.lastname: Razen
authors.affiliation: NUS Business School, National University of Singapore, Singapore 119245.
authors.forename: Hang
authors.initials: H
authors.lastname: Wu