title: An unbiased method to build benchmarking sets for ligand-based virtual screening and its application to GPCRs.
abstract: Benchmarking data sets have become common in recent years for the purpose of virtual screening, though the main focus had been placed on the structure-based virtual screening (SBVS) approaches. Due to the lack of crystal structures, there is great need for unbiased benchmarking sets to evaluate various ligand-based virtual screening (LBVS) methods for important drug targets such as G protein-coupled receptors (GPCRs). To date these ready-to-apply data sets for LBVS are fairly limited, and the direct usage of benchmarking sets designed for SBVS could bring the biases to the evaluation of LBVS. Herein, we propose an unbiased method to build benchmarking sets for LBVS and validate it on a multitude of GPCRs targets. To be more specific, our methods can (1) ensure chemical diversity of ligands, (2) maintain the physicochemical similarity between ligands and decoys, (3) make the decoys dissimilar in chemical topology to all ligands to avoid false negatives, and (4) maximize spatial random distribution of ligands and decoys. We evaluated the quality of our Unbiased Ligand Set (ULS) and Unbiased Decoy Set (UDS) using three common LBVS approaches, with Leave-One-Out (LOO) Cross-Validation (CV) and a metric of average AUC of the ROC curves. Our method has greatly reduced the "artificial enrichment" and "analogue bias" of a published GPCRs benchmarking set, i.e., GPCR Ligand Library (GLL)/GPCR Decoy Database (GDD). In addition, we addressed an important issue about the ratio of decoys per ligand and found that for a range of 30 to 100 it does not affect the quality of the benchmarking set, so we kept the original ratio of 39 from the GLL/GDD.
authors.affiliation: State Key Laboratory of Natural and Biomimetic Drugs, School of Pharmaceutical Sciences, Peking University , Beijing 100191, China.
authors.forename: Jie
authors.initials: J
authors.lastname: Xia
authors.forename: Hongwei
authors.initials: H
authors.lastname: Jin
authors.forename: Zhenming
authors.initials: Z
authors.lastname: Liu
authors.forename: Liangren
authors.initials: L
authors.lastname: Zhang
authors.forename: Xiang Simon
authors.initials: XS
authors.lastname: Wang
mesh_terms: D019985:Benchmarking; D055598:Chemical Phenomena; D062313:Databases, Pharmaceutical; D004353:Drug Evaluation, Preclinical / Q000379:methods*; D006801:Humans; D008024:Ligands; D043562:Receptors, G-Protein-Coupled / Q000819:agonists / Q000037:antagonists & inhibitors / Q000378:metabolism*; D014584:User-Computer Interface
chemical_list: D008024:Ligands; D043562:Receptors, G-Protein-Coupled
references.citation: J Chem Inf Model. 2012 Jul 23;52(7):1757-68
references.pmid: 22587354
references.citation: Nucleic Acids Res. 2008 Jan;36(Database issue):D907-12
references.pmid: 17986454
references.citation: J Chem Inf Model. 2009 Feb;49(2):169-84
references.pmid: 19434821
references.citation: J Chem Inf Comput Sci. 2004 May-Jun;44(3):793-806
references.pmid: 15154744
references.citation: J Chem Inf Model. 2012 Jun 25;52(6):1607-20
references.pmid: 22646988
references.citation: J Chem Inf Model. 2012 Jan 23;52(1):1-6
references.pmid: 22168315
references.citation: Drug Discov Today. 2010 Jun;15(11-12):444-50
references.pmid: 20362693
references.citation: J Med Chem. 2006 Nov 16;49(23):6789-801
references.pmid: 17154509
references.citation: Acta Pharmacol Sin. 2012 Mar;33(3):324-34
references.pmid: 22286917
references.citation: Nature. 2009 Nov 12;462(7270):175-81
references.pmid: 19881490
references.citation: J Comput Aided Mol Des. 2008 Mar-Apr;22(3-4):193-9
references.pmid: 18273555
references.citation: J Comput Aided Mol Des. 2009 Nov;23(11):747-54
references.pmid: 18483766
references.citation: J Comput Aided Mol Des. 2012 Jun;26(6):725-35
references.pmid: 22569592
references.citation: Methods Mol Biol. 2011;672:133-58
references.pmid: 20838967
references.citation: Nat Rev Drug Discov. 2011 Jan;10(1):47-60
references.pmid: 21193867
references.citation: J Chem Inf Model. 2011 Aug 22;51(8):1754-61
references.pmid: 21761904
references.citation: J Chem Inf Model. 2007 Jul-Aug;47(4):1504-19
references.pmid: 17591764
references.citation: Curr Med Chem. 2011;18(17):2543-53
references.pmid: 21568893
references.citation: J Mol Graph Model. 2012 Sep;38:342-53
references.pmid: 23085173
references.citation: Comb Chem High Throughput Screen. 2011 Dec;14(10):872-88
references.pmid: 21843144
references.citation: J Biol Chem. 1981 Jun 10;256(11):5820-6
references.pmid: 6263890
references.citation: J Chem Inf Model. 2008 May;48(5):997-1013
references.pmid: 18470978
references.citation: J Chem Inf Model. 2008 Apr;48(4):704-18
references.pmid: 18380447
references.citation: Drug Discov Today. 2006 Dec;11(23-24):1046-53
references.pmid: 17129822
references.citation: Curr Comput Aided Drug Des. 2010 Mar;6(1):37-49
references.pmid: 20370694
references.citation: Molecules. 2013 Jan 08;18(1):735-56
references.pmid: 23299552
references.citation: J Med Chem. 2012 Jul 26;55(14):6582-94
references.pmid: 22716043
references.citation: Ernst Schering Found Symp Proc. 2006;(2):49-73
references.pmid: 17703577
references.citation: Nat Chem Biol. 2012 Jul 18;8(8):670-3
references.pmid: 22810761
references.citation: J Chem Inf Model. 2011 Oct 24;51(10):2467-73
references.pmid: 21902278
references.citation: J Chem Inf Model. 2012 Apr 23;52(4):867-81
references.pmid: 22435959
references.citation: J Chem Inf Model. 2011 Oct 24;51(10):2650-65
references.pmid: 21774552
references.citation: Nucleic Acids Res. 2009 Jul;37(Web Server issue):W623-33
references.pmid: 19498078
references.citation: J Chem Inf Model. 2012 May 25;52(5):1103-13
references.pmid: 22551340
references.citation: J Med Chem. 2011 Jul 14;54(13):4283-311
references.pmid: 21615150
references.citation: Nat Rev Drug Discov. 2013 Jan;12(1):25-34
references.pmid: 23237917
references.citation: J Comput Aided Mol Des. 2008 Mar-Apr;22(3-4):169-78
references.pmid: 18188508
references.citation: J Chem Inf Model. 2009 Feb;49(2):209-31
references.pmid: 19434824
references.citation: Bioinformatics. 2012 Jun 15;28(12):1661-2
references.pmid: 22539671
references.citation: Nat Rev Drug Discov. 2006 Dec;5(12):993-6
references.pmid: 17139284
references.citation: J Med Chem. 1996 Jul 19;39(15):2887-93
references.pmid: 8709122
references.citation: Trends Pharmacol Sci. 2012 May;33(5):249-60
references.pmid: 22465153
references.citation: Curr Pharm Des. 2007;13(34):3494-504
references.pmid: 18220786
references.citation: J Cheminform. 2009 Aug 25;1:14
references.pmid: 20150995
references.citation: J Comput Aided Mol Des. 2008 Mar-Apr;22(3-4):141-6
references.pmid: 18256892
references.citation: Methods Mol Biol. 2011;672:261-98
references.pmid: 20838973
references.citation: Curr Pharm Des. 2005;11(9):1189-202
references.pmid: 15853666
references.citation: J Chem Inf Model. 2013 Jan 28;53(1):230-40
references.pmid: 23240691
references.citation: Mol Divers. 2008 Feb;12(1):25-40
references.pmid: 18317941
references.citation: J Chem Inf Model. 2011 Feb 28;51(2):196-202
references.pmid: 21207928
references.citation: Chem Biol Drug Des. 2010 Mar;75(3):325-32
references.pmid: 20659113
references.citation: Curr Opin Pharmacol. 2009 Oct;9(5):629-35
references.pmid: 19443270
references.citation: Acta Biochim Pol. 2012;59(4):515-29
references.pmid: 23251911
references.citation: J Chem Inf Model. 2010 Oct 25;50(10):1790-5
references.pmid: 20886834