title: BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology.
abstract: BACKGROUND
  Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions.
  
  
  RESULTS
  Here we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker's yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation.
  
  
  CONCLUSIONS
  This suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from the BioPreDyn-bench website, https://sites.google.com/site/biopredynbenchmarks/ .
authors.affiliation: Bioprocess Engineering Group, IIM-CSIC, Eduardo Cabello 6, Vigo, 36208, Spain. afvillaverde@iim.csic.es.
authors.forename: Alejandro F
authors.initials: AF
authors.lastname: Villaverde
authors.affiliation: Bioprocess Engineering Group, IIM-CSIC, Eduardo Cabello 6, Vigo, 36208, Spain. davidh@iim.csic.es.|European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome Campus, Hinxton, CB10 1SD, Cambridge, UK. davidh@iim.csic.es.
authors.forename: David
authors.initials: D
authors.lastname: Henriques
authors.affiliation: School of Computer Science, Manchester Centre for Integrative Systems Biology, The University of Manchester, 131 Princess Street, Manchester, M1 7DN, UK. kieran.smallbone@manchester.ac.uk.
authors.forename: Kieran
authors.initials: K
authors.lastname: Smallbone
authors.affiliation: Insilico Biotechnology AG, Meitnerstraße 8, Stuttgart, 70563, Germany. sophia.bongard@insilico-biotechnology.com.
authors.forename: Sophia
authors.initials: S
authors.lastname: Bongard
authors.affiliation: Insilico Biotechnology AG, Meitnerstraße 8, Stuttgart, 70563, Germany. joachim.schmid@insilico-biotechnology.com.
authors.forename: Joachim
authors.initials: J
authors.lastname: Schmid
authors.affiliation: EMBL/CRG Research Unit in Systems Biology, Centre for Genomic Regulation (CRG), Dr. Aiguader 88, Barcelona, 08003, Spain. damjan.cicin@crg.eu.|Universitat Pompeu Fabra (UPF), Plaça de la Mercé, 10, Barcelona, 08002, Spain. damjan.cicin@crg.eu.
authors.forename: Damjan
authors.initials: D
authors.lastname: Cicin-Sain
authors.affiliation: EMBL/CRG Research Unit in Systems Biology, Centre for Genomic Regulation (CRG), Dr. Aiguader 88, Barcelona, 08003, Spain. Anton.Crombach@crg.eu.|Universitat Pompeu Fabra (UPF), Plaça de la Mercé, 10, Barcelona, 08002, Spain. Anton.Crombach@crg.eu.
authors.forename: Anton
authors.initials: A
authors.lastname: Crombach
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome Campus, Hinxton, CB10 1SD, Cambridge, UK. saezrodriguez@ebi.ac.uk.
authors.forename: Julio
authors.initials: J
authors.lastname: Saez-Rodriguez
authors.affiliation: Insilico Biotechnology AG, Meitnerstraße 8, Stuttgart, 70563, Germany. klaus.mauch@insilico-biotechnology.com.
authors.forename: Klaus
authors.initials: K
authors.lastname: Mauch
authors.affiliation: Bioprocess Engineering Group, IIM-CSIC, Eduardo Cabello 6, Vigo, 36208, Spain. ebalsa@iim.csic.es.
authors.forename: Eva
authors.initials: E
authors.lastname: Balsa-Canto
authors.affiliation: School of Computer Science, Manchester Centre for Integrative Systems Biology, The University of Manchester, 131 Princess Street, Manchester, M1 7DN, UK. pedro.mendes@manchester.ac.uk.
authors.forename: Pedro
authors.initials: P
authors.lastname: Mendes
authors.affiliation: EMBL/CRG Research Unit in Systems Biology, Centre for Genomic Regulation (CRG), Dr. Aiguader 88, Barcelona, 08003, Spain. yogi.jaeger@crg.eu.|Universitat Pompeu Fabra (UPF), Plaça de la Mercé, 10, Barcelona, 08002, Spain. yogi.jaeger@crg.eu.
authors.forename: Johannes
authors.initials: J
authors.lastname: Jaeger
authors.affiliation: Bioprocess Engineering Group, IIM-CSIC, Eduardo Cabello 6, Vigo, 36208, Spain. julio@iim.csic.es.
authors.forename: Julio R
authors.initials: JR
authors.lastname: Banga
mesh_terms: D000465:Algorithms; D000818:Animals; D019985:Benchmarking; D016466:CHO Cells; D002244:Carbon / Q000378:metabolism; D006224:Cricetinae; D003412:Cricetulus; D004331:Drosophila melanogaster / Q000235:genetics; D004926:Escherichia coli / Q000201:enzymology / Q000235:genetics / Q000378:metabolism; D053263:Gene Regulatory Networks; D023281:Genomics; D007700:Kinetics; D008954:Models, Biological; D012441:Saccharomyces cerevisiae / Q000235:genetics; D015398:Signal Transduction; D012984:Software; D049490:Systems Biology / Q000379:methods*; D014158:Transcription, Genetic
chemical_list: D002244:Carbon