title: Assessing the validity and reproducibility of genome-scale predictions.
abstract: BACKGROUND
  Validation and reproducibility of results is a central and pressing issue in genomics. Several recent embarrassing incidents involving the irreproducibility of high-profile studies have illustrated the importance of this issue and the need for rigorous methods for the assessment of reproducibility.
  
  
  RESULTS
  Here, we describe an existing statistical model that is very well suited to this problem. We explain its utility for assessing the reproducibility of validation experiments, and apply it to a genome-scale study of adenosine deaminase acting on RNA (ADAR)-mediated RNA editing in Drosophila. We also introduce a statistical method for planning validation experiments that will obtain the tightest reproducibility confidence limits, which, for a fixed total number of experiments, returns the optimal number of replicates for the study.
  
  
  BACKGROUND
  Downloadable software and a web service for both the analysis of data from a reproducibility study and for the optimal design of these studies is provided at http://ccmbweb.ccv.brown.edu/reproducibility.html .
authors.affiliation: Center for Computational Molecular Biology and the Division of Applied Mathematics, Brown University, Providence, RI 02912, USA, St. Laurent Institute, 317 New Boston St, Woburn, MA 01801, USA and Department of Molecular Biology, Cell Biology and Biochemistry, Brown University, Providence, RI 02912, USA.
authors.forename: Lauren A
authors.initials: LA
authors.lastname: Sugden
authors.forename: Michael R
authors.initials: MR
authors.lastname: Tackett
authors.forename: Yiannis A
authors.initials: YA
authors.lastname: Savva
authors.forename: William A
authors.initials: WA
authors.lastname: Thompson
authors.forename: Charles E
authors.initials: CE
authors.lastname: Lawrence
mesh_terms: D000243:Adenosine Deaminase; D000818:Animals; D004330:Drosophila / Q000235:genetics; D016678:Genome; D023281:Genomics / Q000379:methods*; D015233:Models, Statistical; D017393:RNA Editing; D015203:Reproducibility of Results; D012984:Software
chemical_list: D000243:Adenosine Deaminase
references.citation: Brief Bioinform. 2009 Sep;10(5):556-68
references.pmid: 19679825
references.citation: Nat Rev Neurosci. 2013 May;14(5):365-76
references.pmid: 23571845
references.citation: Genetics. 2010 Jun;185(2):405-16
references.pmid: 20439781
references.citation: Nat Biotechnol. 2010 Apr;28(4):337-40
references.pmid: 20379180
references.citation: Science. 2010 Aug 6;329(5992):643-8
references.pmid: 20616232
references.citation: Science. 2012 Mar 16;335(6074):1302; author reply 1302
references.pmid: 22422963
references.citation: RNA. 2006 Nov;12(11):1922-32
references.pmid: 17018572
references.citation: Nat Rev Drug Discov. 2011 Aug 31;10(9):712
references.pmid: 21892149
references.citation: Nat Genet. 2011 Dec 27;44(1):3-5
references.pmid: 22200769
references.citation: Genome Biol. 2002;3(5):research0022
references.pmid: 12049663
references.citation: PLoS Biol. 2008 Feb;6(2):e27
references.pmid: 18271625
references.citation: Annu Rev Biochem. 2010;79:321-49
references.pmid: 20192758
references.citation: Proc Natl Acad Sci U S A. 2001 Jul 31;98(16):8961-5
references.pmid: 11470909
references.citation: BMC Syst Biol. 2011;5 Suppl 2:S15
references.pmid: 22784572
references.citation: Nat Methods. 2011 Dec 25;9(2):189-94
references.pmid: 22198342
references.citation: Nature. 2012 Jul 25;487(7408):427-8
references.pmid: 22836983
references.citation: BMC Genomics. 2004 Nov 08;5:87
references.pmid: 15533245
references.citation: Nature. 2012 Mar 28;483(7391):531-3
references.pmid: 22460880
references.citation: Pharmacogenomics. 2009 Feb;10(2):191-201
references.pmid: 19207020
references.citation: PLoS Genet. 2012;8(3):e1002600
references.pmid: 22479196
references.citation: Nat Biotechnol. 2012 Sep;30(9):806
references.pmid: 22965029
references.citation: Nat Genet. 2008 May;40(5):638-45
references.pmid: 18372903
references.citation: Nat Struct Mol Biol. 2013 Nov;20(11):1333-9
references.pmid: 24077224
references.citation: Science. 2012 Mar 16;335(6074):1302; author reply 1302
references.pmid: 22422964
references.citation: EMBO J. 1991 Nov;10(11):3523-32
references.pmid: 1915306
references.citation: J Bioinform Comput Biol. 2006 Jun;4(3):693-708
references.pmid: 16960970
references.citation: Nature. 2012 Dec 13;492(7428):180-1
references.pmid: 23235861
references.citation: Bioinformatics. 2002 Nov;18(11):1462-9
references.pmid: 12424117
references.citation: Bioinformatics. 2012 Jul 1;28(13):1721-8
references.pmid: 22563066
references.citation: Nature. 2012 May 16;485(7398):298-300
references.pmid: 22596136
references.citation: Nature. 2011 Sep 28;477(7366):511
references.pmid: 21956292
references.citation: Cell. 2000 Aug 18;102(4):437-49
references.pmid: 10966106
references.citation: Nat Genet. 2008 Aug;40(8):955-62
references.pmid: 18587394
references.citation: Science. 2011 Jul 1;333(6038):53-8
references.pmid: 21596952
references.citation: Bioinformatics. 2009 Jul 1;25(13):1662-8
references.pmid: 19417058
references.citation: Bioinformatics. 2002 Dec;18(12):1609-16
references.pmid: 12490445
references.citation: Cell. 1988 Dec 23;55(6):1089-98
references.pmid: 3203381
references.citation: BMC Genomics. 2005 May 20;6:77
references.pmid: 15907200
references.citation: PLoS Genet. 2010 Jan 15;6(1):e1000814
references.pmid: 20084099
references.citation: Nat Genet. 2007 Dec;39(12):1512-6
references.pmid: 17994019
references.citation: Biostatistics. 2009 Apr;10(2):297-309
references.pmid: 19075295
references.citation: Nature. 2010 Jul 8;466(7303):243-7
references.pmid: 20555325
references.citation: BMC Bioinformatics. 2004 Aug 31;5:119
references.pmid: 15339345
references.citation: BMC Bioinformatics. 2006 Mar 02;7:106
references.pmid: 16512900
references.citation: PLoS One. 2013 Apr 24;8(4):e56515
references.pmid: 23637732
references.citation: Proc Natl Acad Sci U S A. 2008 Jan 15;105(2):617-22
references.pmid: 18174335
references.citation: Genome Res. 2011 Feb;21(2):182-92
references.pmid: 21177961
references.citation: Science. 2003 Aug 8;301(5634):832-6
references.pmid: 12907802
references.citation: Nat Methods. 2009 Jun;6(6):423-30
references.pmid: 19448641
references.citation: Science. 2012 Mar 16;335(6074):1302; author reply 1302
references.pmid: 22422962
references.citation: Nat Genet. 2009 Feb;41(2):149-55
references.pmid: 19174838
references.citation: Bioinformatics. 2003 Aug 12;19(12):1477-83
references.pmid: 12912827
references.citation: Nat Rev Genet. 2010 Oct;11(10):733-9
references.pmid: 20838408
references.citation: Nature. 2013 Apr 4;496(7443):7
references.pmid: 23552910
references.citation: J Biol. 2009;8(1):3
references.pmid: 19222867
references.citation: Nat Biotechnol. 2006 Sep;24(9):1151-61
references.pmid: 16964229
references.citation: J Biomol Screen. 2010 Aug;15(7):735-47
references.pmid: 20625183
references.citation: Nat Biotechnol. 2006 Jul;24(7):832-40
references.pmid: 16823376