title: GRID-based three-dimensional pharmacophores II: PharmBench, a benchmark data set for evaluating pharmacophore elucidation methods.
abstract: To date, published pharmacophore elucidation approaches typically use a handful of data sets for validation: here, we have assembled a data set for 81 targets, containing 960 ligands aligned using their cocrystallized protein targets, to provide the experimental "gold standard". The two-dimensional structures are also assembled to remove conformational bias; an ideal method would be able to take these structures as input, find the common features, and reproduce the bioactive conformations and their alignments to correspond with the X-ray-determined gold standard alignments. Here we present this data set and describe three objective measures to evaluate performance: the ability to identify the bioactive conformation, the ability to identify and correctly align this conformation for 50% of the molecules in each data set, and the pharmacophoric field similarity. We have applied this validation methodology to our pharmacophore elucidation method FLAPpharm, that is published in the first paper of this series and discuss the limitations of the data set and objective success criteria. Starting from two-dimensional structures and producing unbiased models, FLAPpharm was able to identify the bioactive conformations for 67% of the ligands and also to produce successful models according to the second metric for 67% of the Pharmbench data sets. Inspection of the unsuccessful models highlighted the limitation of this root mean square (rms)-derived metric, since many were found to be pharmacophorically reasonable, increasing the overall success rate to 83%. The PharmBench data set is available at http://www.moldiscovery.com/PharmBench , along with a web service to enable users to score model alignments coming from external methods in the same way that we have presented here and, therefore, establishes a pharmacophore elucidation benchmark data set available to be used by the community.
authors.affiliation: Molecular Discovery Limited, 215 Marsh Road, Pinner, Middlesex, London HA5 5NE, United Kingdom. simon@moldiscovery.com
authors.forename: Simon
authors.initials: S
authors.lastname: Cross
authors.forename: Francesco
authors.initials: F
authors.lastname: Ortuso
authors.forename: Massimo
authors.initials: M
authors.lastname: Baroni
authors.forename: Giosuè
authors.initials: G
authors.lastname: Costa
authors.forename: Simona
authors.initials: S
authors.lastname: Distinto
authors.forename: Federica
authors.initials: F
authors.lastname: Moraca
authors.forename: Stefano
authors.initials: S
authors.lastname: Alcaro
authors.forename: Gabriele
authors.initials: G
authors.lastname: Cruciani
mesh_terms: D000465:Algorithms; D001665:Binding Sites; D001688:Biological Products / Q000737:chemistry*; D003198:Computer Simulation; D018360:Crystallography, X-Ray; D015195:Drug Design; D006801:Humans; D057927:Hydrophobic and Hydrophilic Interactions; D008024:Ligands; D008958:Models, Molecular; D011506:Proteins / Q000737:chemistry*; D012107:Research Design; D013329:Structure-Activity Relationship
chemical_list: D001688:Biological Products; D008024:Ligands; D011506:Proteins