title: Where next for the reproducibility agenda in computational biology?
abstract: The concept of reproducibility is a foundation of the scientific method. With the arrival of fast and powerful computers over the last few decades, there has been an explosion of results based on complex computational analyses and simulations. The reproducibility of these results has been addressed mainly in terms of exact replicability or numerical equivalence, ignoring the wider issue of the reproducibility of conclusions through equivalent, extended or alternative methods.
  
  
  
  We use case studies from our own research experience to illustrate how concepts of reproducibility might be applied in computational biology. Several fields have developed 'minimum information' checklists to support the full reporting of computational simulations, analyses and results, and standardised data formats and model description languages can facilitate the use of multiple systems to address the same research question. We note the importance of defining the key features of a result to be reproduced, and the expected agreement between original and subsequent results. Dynamic, updatable tools for publishing methods and results are becoming increasingly common, but sometimes come at the cost of clear communication. In general, the reproducibility of computational research is improving but would benefit from additional resources and incentives.
  
  
  
  We conclude with a series of linked recommendations for improving reproducibility in computational biology through communication, policy, education and research practice. More reproducible research will lead to higher quality conclusions, deeper understanding and more valuable knowledge.
authors.affiliation: Centre for Maths and Physics in the Life Sciences and Experimental Biology, University College London, Physics Building, Gower Place, London, WC1E 6BT, UK. joanna.lewis@imperial.ac.uk.|NIHR Health Protection Research Unit in Modelling Methodology, Department of Infectious Disease Epidemiology, Imperial College London, St Mary's Campus, Norfolk Place, London, W2 1PG, UK. joanna.lewis@imperial.ac.uk.
authors.forename: Joanna
authors.initials: J
authors.lastname: Lewis
authors.affiliation: UCL Cancer Institute, University College London, 72 Huntley St, London, WC1E 6DD, UK.
authors.forename: Charles E
authors.initials: CE
authors.lastname: Breeze
authors.affiliation: Department of Genetics, University of Cambridge, Downing Street, Cambridge, CB2 3EH, UK.
authors.forename: Jane
authors.initials: J
authors.lastname: Charlesworth
authors.affiliation: Department of Mathematics, University of Auckland, Auckland, 1142, New Zealand.|Department of Engineering Science, University of Auckland, Auckland, 1142, New Zealand.
authors.forename: Oliver J
authors.initials: OJ
authors.lastname: Maclaren
authors.affiliation: Department of Computer Science, University of Oxford, Wolfson Building, Parks Road, Oxford, OX1 3QD, UK.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Cooper
mesh_terms: D019295:Computational Biology / Q000379:methods*; D015203:Reproducibility of Results; D012984:Software
keywords: Communication; Education; Extensibility; Policy; Replicability; Reproducibility