title: Performance of Deep and Shallow Neural Networks, the Universal Approximation Theorem, Activity Cliffs, and QSAR.
abstract: Neural networks have generated valuable Quantitative Structure-Activity/Property Relationships (QSAR/QSPR) models for a wide variety of small molecules and materials properties. They have grown in sophistication and many of their initial problems have been overcome by modern mathematical techniques. QSAR studies have almost always used so-called "shallow" neural networks in which there is a single hidden layer between the input and output layers. Recently, a new and potentially paradigm-shifting type of neural network based on Deep Learning has appeared. Deep learning methods have generated impressive improvements in image and voice recognition, and are now being applied to QSAR and QSAR modelling. This paper describes the differences in approach between deep and shallow neural networks, compares their abilities to predict the properties of test sets for 15 large drug data sets (the kaggle set), discusses the results in terms of the Universal Approximation theorem for neural networks, and describes how DNN may ameliorate or remove troublesome "activity cliffs" in QSAR data sets.
authors: CSIRO Manufacturing, Clayton, 3168, Australia.|Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, 3052, Australia.|Latrobe Institute for Molecular Science, Latrobe University, Bundoora, 3082, Australia.|School of Chemical and Physical Science, Flinders University, Bedford Park, 5042, Australia.
authors: David A
authors: DA
authors: Winkler
authors: CSIRO Manufacturing, Clayton, 3168, Australia.
authors: Tu C
authors: TC
authors: Le
mesh_terms: D015195:Drug Design; D000069550:Machine Learning; D062105:Molecular Docking Simulation / Q000379:methods* / Q000592:standards; D021281:Quantitative Structure-Activity Relationship; D054852:Small Molecule Libraries / Q000737:chemistry / Q000494:pharmacology
chemical_list: D054852:Small Molecule Libraries
keywords: Bayesian regularized neural network; activity cliff; deep learning; deep neural network; shallow neural network; universal approximation theorem