Boosting: an ensemble learning tool for compound classification and QSAR modeling.
A classification and regression tool, J. H. Friedman's Stochastic Gradient Boosting (SGB), is applied to predicting a compound's quantitative or categorical biological activity based on a quantitative description of the compound's molecular structure. Stochastic Gradient Boosting is a procedure for building a sequence of models, for instance regression trees (as in this paper), whose outputs are combined to form a predicted quantity, either an estimate of the biological activity, or a class label to which a molecule belongs. In particular, the SGB procedure builds a model in a stage-wise manner by fitting each tree to the gradient of a loss function: e.g., squared error for regression and binomial log-likelihood for classification. The values of the gradient are computed for each sample in the training set, but only a random sample of these gradients is used at each stage. (Friedman showed that the well-known boosting algorithm, AdaBoost of Freund and Schapire, could be considered as a particular case of SGB.) The SGB method is used to analyze 10 cheminformatics data sets, most of which are publicly available. The results show that SGB's performance is comparable to that of Random Forest, another ensemble learning method, and are generally competitive with or superior to those of other QSAR methods. The use of SGB's variable importance with partial dependence plots for model interpretation is also illustrated.
Svetnik
Vladimir
V
Biometrics Research and Molecular Systems, Merck Research Laboratories, P.O. Box 2000, Rahway, New Jersey 07065, USA. vladimir_svetnik@merck.com
Wang
Ting
T
Tong
Christopher
C
Liaw
Andy
A
Sheridan
Robert P
RP
Song
Qinghua
Q
D020168:ATP Binding Cassette Transporter, Subfamily B, Member 1 / Q000378:metabolism; D000465:Algorithms; D001812:Blood-Brain Barrier; D042846:CDC2-CDC28 Kinases / Q000037:antagonists & inhibitors; D051357:Cyclin-Dependent Kinase 2; D016861:Cyclooxygenase Inhibitors / Q000737:chemistry; D018432:Drug Resistance, Multiple / Q000187:drug effects; D004967:Estrogens / Q000378:metabolism; D008958:Models, Molecular; D021281:Quantitative Structure-Activity Relationship; D017448:Receptors, Dopamine D2 / Q000378:metabolism; D013269:Stochastic Processes
D020168:ATP Binding Cassette Transporter, Subfamily B, Member 1; D016861:Cyclooxygenase Inhibitors; D004967:Estrogens; D017448:Receptors, Dopamine D2; D042846:CDC2-CDC28 Kinases; D051357:Cyclin-Dependent Kinase 2