title: Evaluation of the performance of 3D virtual screening protocols: RMSD comparisons, enrichment assessments, and decoy selection--what can we learn from earlier mistakes?
abstract: Within the last few years a considerable amount of evaluative studies has been published that investigate the performance of 3D virtual screening approaches. Thereby, in particular assessments of protein-ligand docking are facing remarkable interest in the scientific community. However, comparing virtual screening approaches is a non-trivial task. Several publications, especially in the field of molecular docking, suffer from shortcomings that are likely to affect the significance of the results considerably. These quality issues often arise from poor study design, biasing, by using improper or inexpressive enrichment descriptors, and from errors in interpretation of the data output. In this review we analyze recent literature evaluating 3D virtual screening methods, with focus on molecular docking. We highlight problematic issues and provide guidelines on how to improve the quality of computational studies. Since 3D virtual screening protocols are in general assessed by their ability to discriminate between active and inactive compounds, we summarize the impact of the composition and preparation of test sets on the outcome of evaluations. Moreover, we investigate the significance of both classic enrichment parameters and advanced descriptors for the performance of 3D virtual screening methods. Furthermore, we review the significance and suitability of RMSD as a measure for the accuracy of protein-ligand docking algorithms and of conformational space sub sampling algorithms.
authors.affiliation: Inte:Ligand Software-Entwicklungs- und Consulting GmbH, Clemens Maria Hofbauer-Gasse 6, 2344, Maria Enzersdorf, Austria.
authors.forename: Johannes
authors.initials: J
authors.lastname: Kirchmair
authors.forename: Patrick
authors.initials: P
authors.lastname: Markt
authors.forename: Simona
authors.initials: S
authors.lastname: Distinto
authors.forename: Gerhard
authors.initials: G
authors.lastname: Wolber
authors.forename: Thierry
authors.initials: T
authors.lastname: Langer
mesh_terms: D017076:Computer-Aided Design; D016503:Drug Delivery Systems / Q000379:methods*; D015195:Drug Design; D005069:Evaluation Studies as Topic; D008024:Ligands; D011506:Proteins / Q000378:metabolism; D012984:Software
chemical_list: D008024:Ligands; D011506:Proteins