title: Benchmarking Procedures for High-Throughput Context Specific Reconstruction Algorithms.
abstract: Recent progress in high-throughput data acquisition has shifted the focus from data generation to processing and understanding of how to integrate collected information. Context specific reconstruction based on generic genome scale models like ReconX or HMR has the potential to become a diagnostic and treatment tool tailored to the analysis of specific individuals. The respective computational algorithms require a high level of predictive power, robustness and sensitivity. Although multiple context specific reconstruction algorithms were published in the last 10 years, only a fraction of them is suitable for model building based on human high-throughput data. Beside other reasons, this might be due to problems arising from the limitation to only one metabolic target function or arbitrary thresholding. This review describes and analyses common validation methods used for testing model building algorithms. Two major methods can be distinguished: consistency testing and comparison based testing. The first is concerned with robustness against noise, e.g., missing data due to the impossibility to distinguish between the signal and the background of non-specific binding of probes in a microarray experiment, and whether distinct sets of input expressed genes corresponding to i.e., different tissues yield distinct models. The latter covers methods comparing sets of functionalities, comparison with existing networks or additional databases. We test those methods on several available algorithms and deduce properties of these algorithms that can be compared with future developments. The set of tests performed, can therefore serve as a benchmarking procedure for future algorithms.
authors.affiliation: Systems Biology Group, Life Sciences Research Unit, University of Luxembourg, Luxembourg Luxembourg.
authors.forename: Maria P
authors.initials: MP
authors.lastname: Pacheco
authors.affiliation: Systems Biology Group, Life Sciences Research Unit, University of Luxembourg, LuxembourgLuxembourg; Department of Physics, Institute of Complex Systems and Mathematical Biology, University of AberdeenAberdeen, UK.
authors.forename: Thomas
authors.initials: T
authors.lastname: Pfau
authors.affiliation: Systems Biology Group, Life Sciences Research Unit, University of Luxembourg, Luxembourg Luxembourg.
authors.forename: Thomas
authors.initials: T
authors.lastname: Sauter
keywords: benchmarking; constraint-based modeling; metabolic networks and pathways; metabolic reconstruction; tissue specific networks; validation
references.citation: J Nutr. 1984 Sep;114(9):1664-74
references.pmid: 6088736
references.citation: BMC Genomics. 2015 Oct 19;16:809
references.pmid: 26480823
references.citation: Genomics. 2005 Aug;86(2):127-41
references.pmid: 15950434
references.citation: BMC Syst Biol. 2012 Jun 19;6:73
references.pmid: 22713172
references.citation: Science. 2015 Jan 23;347(6220):1260419
references.pmid: 25613900
references.citation: BMC Syst Biol. 2012 Dec 13;6:153
references.pmid: 23234303
references.citation: Mol Syst Biol. 2013;9:649
references.pmid: 23511207
references.citation: PLoS Comput Biol. 2014 Apr 24;10(4):e1003580
references.pmid: 24762745
references.citation: Mol Syst Biol. 2014 Mar 19;10:721
references.pmid: 24646661
references.citation: PLoS Comput Biol. 2009 Aug;5(8):e1000489
references.pmid: 19714220
references.citation: Nat Methods. 2007 Nov;4(11):911-3
references.pmid: 17906632
references.citation: Proc Natl Acad Sci U S A. 2007 Feb 6;104(6):1777-82
references.pmid: 17267599
references.citation: Bioinformatics. 2010 Dec 15;26(24):3140-2
references.pmid: 21081510
references.citation: PLoS Comput Biol. 2014 Jan;10(1):e1003424
references.pmid: 24453953
references.citation: Nat Biotechnol. 2013 May;31(5):419-25
references.pmid: 23455439
references.citation: PLoS Comput Biol. 2008 May 16;4(5):e1000082
references.pmid: 18483554
references.citation: Nucleic Acids Res. 2013 Jan;41(Database issue):D764-72
references.pmid: 23203881
references.citation: Nat Protoc. 2011 Aug 04;6(9):1290-307
references.pmid: 21886097
references.citation: Nat Biotechnol. 2008 Sep;26(9):1003-10
references.pmid: 18711341
references.citation: Nucleic Acids Res. 2011 Jan;39(Database issue):D1011-5
references.pmid: 21177656
references.citation: Nucleic Acids Res. 2002 Jan 1;30(1):207-10
references.pmid: 11752295
references.citation: Mol Syst Biol. 2010 Sep 7;6:411
references.pmid: 20823849
references.citation: Metab Eng. 2004 Oct;6(4):285-93
references.pmid: 15491858
references.citation: Integr Biol (Camb). 2015 Aug;7(8):859-68
references.pmid: 25730289
references.citation: Mol Syst Biol. 2009;5:263
references.pmid: 19401675
references.citation: Nucleic Acids Res. 2013 Jan;41(Database issue):D991-5
references.pmid: 23193258
references.citation: Mol Syst Biol. 2010 Sep 7;6:401
references.pmid: 20823844
references.citation: Front Plant Sci. 2014 Sep 19;5:491
references.pmid: 25285097
references.citation: PLoS One. 2015 Jul 09;10(7):e0131875
references.pmid: 26158726
references.citation: PLoS Comput Biol. 2012;8(5):e1002518
references.pmid: 22615553
references.citation: Mol Syst Biol. 2011 Jun 21;7:501
references.pmid: 21694718
references.citation: J Biotechnol. 2014 Aug 20;184:172-8
references.pmid: 24907410
references.citation: Elife. 2014 Nov 21;3:null
references.pmid: 25415239
references.citation: Biochim Biophys Acta. 2006 Jul;1763(7):723-36
references.pmid: 16839620
references.citation: Nat Commun. 2014;5:3083
references.pmid: 24419221