CAUTION: popular "benchmark" data sets do not distinguish the merits of 3D QSAR methods.
The quality of 3D QSAR models obtained using extremely simple descriptors was examined for nine popular data sets, including the well-known set of 31 steroids, which for 20 years has been the standard for benchmarking 3D QSAR methods. The atomic numbers of atoms coinciding with vertices of the molecular alignment as well as binary descriptors indicating the occupancy of those vertices were compared to models obtained using SAMFA descriptors, which have been shown to yield models statistically indistinguishable from CoMFA. For most data sets, only a minor loss in model performance was observed, even for the occupancy descriptors, where all chemical information is neglected. As a further simplification, models were fitted using descriptors from just a few atomic positions occupied in the majority of active or inactive compounds. No further loss in performance was observed, even though in one case descriptors from a single atomic position were used, and in all cases the number of atomic positions required was fewer than twelve. The resulting models suggest that simply filling space at a few key atomic positions is responsible for enhanced activity. At least for the steroids, this finding is at odds with known SAR and binding interactions with the relevant receptor. Using a simulated data set, we illustrate that this paradoxical outcome is a symptom of having too few observations to describe the response in a data set. It is concluded that none of the nine data sets examined can reliably distinguish the merits of different 3D QSAR descriptors and that they should not be used for this purpose. We advocate the use of simulated data, instead.
Manchester
John
J
AstraZeneca Pharmaceuticals LLP, Waltham, Massachusetts 02451, USA.
Czermiński
Ryszard
R
D000818:Animals; D019985:Benchmarking / Q000379:methods*; D006801:Humans; D051379:Mice; D008958:Models, Molecular; D008968:Molecular Conformation; D011506:Proteins / Q000737:chemistry / Q000378:metabolism; D011786:Quality Control; D021281:Quantitative Structure-Activity Relationship; D013256:Steroids / Q000737:chemistry / Q000378:metabolism
D011506:Proteins; D013256:Steroids