title: Greedy and Linear Ensembles of Machine Learning Methods Outperform Single Approaches for QSPR Regression Problems.
abstract: The application of Machine Learning to cheminformatics is a large and active field of research, but there exist few papers which discuss whether ensembles of different Machine Learning methods can improve upon the performance of their component methodologies. Here we investigated a variety of methods, including kernel-based, tree, linear, neural networks, and both greedy and linear ensemble methods. These were all tested against a standardised methodology for regression with data relevant to the pharmaceutical development process. This investigation focused on QSPR problems within drug-like chemical space. We aimed to investigate which methods perform best, and how the 'wisdom of crowds' principle can be applied to ensemble predictors. It was found that no single method performs best for all problems, but that a dynamic, well-structured ensemble predictor would perform very well across the board, usually providing an improvement in performance over the best single method. Its use of weighting factors allows the greedy ensemble to acquire a bigger contribution from the better performing models, and this helps the greedy ensemble generally to outperform the simpler linear ensemble. Choice of data preprocessing methodology was found to be crucial to performance of each method too.
authors.affiliation: Biomedical Sciences Research Complex and EaStCHEM School of Chemistry, Purdie Building, University of St Andrews, North Haugh, St Andrews, Scotland, KY16 9ST, UK.|Current address: EaStCHEM, School of Chemistry, University of Edinburgh, King's Buildings, West Mains Rd, Edinburgh, Scotland, EH9 3JJ, UK.
authors.forename: William
authors.initials: W
authors.lastname: Kew
authors.affiliation: Biomedical Sciences Research Complex and EaStCHEM School of Chemistry, Purdie Building, University of St Andrews, North Haugh, St Andrews, Scotland, KY16 9ST, UK. jbom@st-andrews.ac.uk, w.kew@sms.ed.ac.uk.
authors.forename: John B O
authors.initials: JB
authors.lastname: Mitchell
mesh_terms: D001330:Electronic Data Processing; D000069550:Machine Learning; D004364:Pharmaceutical Preparations / Q000737:chemistry*
chemical_list: D004364:Pharmaceutical Preparations
keywords: Greedy ensembles; Linear ensembles; Machine Learning; Quantitative structure-property relationships