title: Towards agile large-scale predictive modelling in drug discovery with flow-based programming design principles.
abstract: Predictive modelling in drug discovery is challenging to automate as it often contains multiple analysis steps and might involve cross-validation and parameter tuning that create complex dependencies between tasks. With large-scale data or when using computationally demanding modelling methods, e-infrastructures such as high-performance or cloud computing are required, adding to the existing challenges of fault-tolerant automation. Workflow management systems can aid in many of these challenges, but the currently available systems are lacking in the functionality needed to enable agile and flexible predictive modelling. We here present an approach inspired by elements of the flow-based programming paradigm, implemented as an extension of the Luigi system which we name SciLuigi. We also discuss the experiences from using the approach when modelling a large set of biochemical interactions using a shared computer cluster.Graphical abstract.
authors.affiliation: Department of Pharmaceutical Biosciences, Uppsala University, Box 591, 751 24 Uppsala, Sweden.
authors.forename: Samuel
authors.initials: S
authors.lastname: Lampa
authors.affiliation: Department of Pharmaceutical Biosciences, Uppsala University, Box 591, 751 24 Uppsala, Sweden.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Alvarsson
authors.affiliation: Department of Pharmaceutical Biosciences, Uppsala University, Box 591, 751 24 Uppsala, Sweden ; Science for Life Laboratory, Uppsala University, Box 3037, 750 03 Uppsala, Sweden.
authors.forename: Ola
authors.identifier: 0000-0002-8083-2864
authors.initials: O
authors.lastname: Spjuth
keywords: Drug discovery; Flow-based programming; Machine learning; Predictive modelling; Workflows