title: On the reproducibility of science: unique identification of research resources in the biomedical literature.
abstract: Scientific reproducibility has been at the forefront of many news stories and there exist numerous initiatives to help address this problem. We posit that a contributor is simply a lack of specificity that is required to enable adequate research reproducibility. In particular, the inability to uniquely identify research resources, such as antibodies and model organisms, makes it difficult or impossible to reproduce experiments even where the science is otherwise sound. In order to better understand the magnitude of this problem, we designed an experiment to ascertain the "identifiability" of research resources in the biomedical literature. We evaluated recent journal articles in the fields of Neuroscience, Developmental Biology, Immunology, Cell and Molecular Biology and General Biology, selected randomly based on a diversity of impact factors for the journals, publishers, and experimental method reporting guidelines. We attempted to uniquely identify model organisms (mouse, rat, zebrafish, worm, fly and yeast), antibodies, knockdown reagents (morpholinos or RNAi), constructs, and cell lines. Specific criteria were developed to determine if a resource was uniquely identifiable, and included examining relevant repositories (such as model organism databases, and the Antibody Registry), as well as vendor sites. The results of this experiment show that 54% of resources are not uniquely identifiable in publications, regardless of domain, journal impact factor, or reporting requirements. For example, in many cases the organism strain in which the experiment was performed or antibody that was used could not be identified. Our results show that identifiability is a serious problem for reproducibility. Based on these results, we provide recommendations to authors, reviewers, journal editors, vendors, and publishers. Scientific efficiency and reproducibility depend upon a research-wide improvement of this substantial problem in science today.
authors.affiliation: Ontology Development Group, Library, Oregon Health & Science University , Portland, OR , USA.
authors.forename: Nicole A
authors.initials: NA
authors.lastname: Vasilevsky
authors.forename: Matthew H
authors.initials: MH
authors.lastname: Brush
authors.forename: Holly
authors.initials: H
authors.lastname: Paddock
authors.forename: Laura
authors.initials: L
authors.lastname: Ponting
authors.forename: Shreejoy J
authors.initials: SJ
authors.lastname: Tripathy
authors.forename: Gregory M
authors.initials: GM
authors.lastname: Larocca
authors.forename: Melissa A
authors.initials: MA
authors.lastname: Haendel
keywords: Antibodies; Cell lines; Constructs; Knockdown reagents; Materials and Methods; Model organisms; Scientific reproducibility
references.citation: BMC Biotechnol. 2013 Jan 14;13:2
references.pmid: 23311978
references.citation: Nat Genet. 2012 Jan 27;44(2):121-6
references.pmid: 22281772
references.citation: EMBO Rep. 2008 Jan;9(1):2
references.pmid: 18174889
references.citation: Nature. 1968 Feb 24;217(5130):750-1
references.pmid: 5641128
references.citation: J Mol Histol. 2013 Jun;44(3):249-57
references.pmid: 23307302
references.citation: Biotechniques. 2007 Nov;43(5):575, 577-8, 581-2 passim
references.pmid: 18072586
references.citation: Nucleic Acids Res. 2013 Jan;41(Database issue):D885-91
references.pmid: 23175610
references.citation: Epilepsy Res. 2009 Dec;87(2-3):234-46
references.pmid: 19833479
references.citation: In Vitro Cell Dev Biol Anim. 2004 Jul-Aug;40(7):211-5
references.pmid: 15638703
references.citation: Nucleic Acids Res. 2012 Jan;40(Database issue):D735-41
references.pmid: 22067452
references.citation: PLoS Biol. 2012;10(5):e1001339
references.pmid: 22666180
references.citation: Nucleic Acids Res. 2010 Jan;38(Database issue):D607-12
references.pmid: 19884130
references.citation: Exp Brain Res. 1985;59(3):625-8
references.pmid: 4029329
references.citation: J Biomed Semantics. 2012 Apr 24;3 Suppl 1:S1
references.pmid: 22541592
references.citation: Proc Natl Acad Sci U S A. 2000 Sep 26;97(20):11038-43
references.pmid: 11005875
references.citation: Nature. 2012 Mar 28;483(7391):531-3
references.pmid: 22460880
references.citation: J Lab Autom. 2013 Aug;18(4):264-8
references.pmid: 23592569
references.citation: Infect Immun. 2011 Oct;79(10):3855-9
references.pmid: 21825063
references.citation: PLoS One. 2009 Nov 30;4(11):e7824
references.pmid: 19956596
references.citation: Database (Oxford). 2013 Apr 19;2013:bat015
references.pmid: 23603846
references.citation: J Cell Sci. 2001 Dec;114(Pt 24):4557-65
references.pmid: 11792820
references.citation: Nat Biotechnol. 2010 Jul;28(7):650-3
references.pmid: 20622827
references.citation: Gigascience. 2013 May 01;2(1):7
references.pmid: 23634721
references.citation: Database (Oxford). 2012 Mar 20;2012:bar067
references.pmid: 22434835
references.citation: Nat Genet. 2000 Oct;26(2):216-20
references.pmid: 11017081
references.citation: J Comp Neurol. 1987 Aug 8;262(2):227-41
references.pmid: 3624553
references.citation: Database (Oxford). 2012 Mar 20;2012:bas005
references.pmid: 22434839
references.citation: Int J Cancer. 2010 Jul 1;127(1):1-8
references.pmid: 20143388
references.citation: Nucleic Acids Res. 2013 Jan;41(Database issue):D751-7
references.pmid: 23125371
references.citation: BMC Dev Biol. 2013 Jan 05;13:1
references.pmid: 23289785
references.citation: PLoS One. 2009 Jun 25;4(6):e6040
references.pmid: 19557180
references.citation: Nucleic Acids Res. 2011 Jan;39(Database issue):D822-9
references.pmid: 21036866