title: Enhancing reproducibility: Failures from Reproducibility Initiatives underline core challenges.
abstract: Efforts to address reproducibility concerns in biomedical research include: initiatives to improve journal publication standards and peer review; increased attention to publishing methodological details that enable experiments to be reconstructed; guidelines on standards for study design, implementation, analysis and execution; meta-analyses of multiple studies within a field to synthesize a common conclusion and; the formation of consortia to adopt uniform protocols and internally reproduce data. Another approach to addressing reproducibility are Reproducibility Initiatives (RIs), well-intended, high-profile, systematically peer-vetted initiatives that are intended to replace the traditional process of scientific self-correction. Outcomes from the RIs reported to date have questioned the usefulness of this approach, particularly when the RI outcome differs from other independent self-correction studies that have reproduced the original finding. As a failed RI attempt is a single outcome distinct from the original study, it cannot provide any definitive conclusions necessitating additional studies that the RI approach has neither the ability nor intent of conducting making it a questionable replacement for self-correction. A failed RI attempt also has the potential to damage the reputation of the author of the original finding. Reproduction is frequently confused with replication, an issue that is more than semantic with the former denoting "similarity" and the latter an "exact copy" - an impossible outcome in research because of known and unknown technical, environmental and motivational differences between the original and reproduction studies. To date, the RI framework has negatively impacted efforts to improve reproducibility, confounding attempts to determine whether a research finding is real.
authors.affiliation: Gladstone Institutes, San Francisco, CA, United States.
authors.forename: Kevin
authors.initials: K
authors.lastname: Mullane
authors.affiliation: Department of Pharmacology, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States. Electronic address: rivoli1635@comcast.net.
authors.forename: Michael
authors.initials: M
authors.lastname: Williams
mesh_terms: D000818:Animals; D035843:Biomedical Research / Q000941:ethics / Q000266:history / Q000592:standards* / Q000639:trends; D019317:Evidence-Based Medicine / Q000266:history* / Q000592:standards / Q000639:trends; D017408:Guidelines as Topic; D049669:History, 16th Century; D049670:History, 17th Century; D049671:History, 18th Century; D049672:History, 19th Century; D049673:History, 20th Century; D049674:History, 21st Century; D006801:Humans; D015201:Meta-Analysis as Topic; D017712:Peer Review, Research / Q000592:standards / Q000639:trends; D010506:Periodicals as Topic / Q000592:standards / Q000639:trends; D011786:Quality Control; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards / Q000639:trends; D015871:Scientific Misconduct / Q000941:ethics / Q000266:history
keywords: Peer review; Reproducibility; Scientific self-replication