title: Predictiveness curves in virtual screening.
abstract: BACKGROUND
  In the present work, we aim to transfer to the field of virtual screening the predictiveness curve, a metric that has been advocated in clinical epidemiology. The literature describes the use of predictiveness curves to evaluate the performances of biological markers to formulate diagnoses, prognoses and assess disease risks, assess the fit of risk models, and estimate the clinical utility of a model when applied to a population. Similarly, we use logistic regression models to calculate activity probabilities related to the scores that the compounds obtained in virtual screening experiments. The predictiveness curve can provide an intuitive and graphical tool to compare the predictive power of virtual screening methods.
  
  
  RESULTS
  Similarly to ROC curves, predictiveness curves are functions of the distribution of the scores and provide a common scale for the evaluation of virtual screening methods. Contrarily to ROC curves, the dispersion of the scores is well described by predictiveness curves. This property allows the quantification of the predictive performance of virtual screening methods on a fraction of a given molecular dataset and makes the predictiveness curve an efficient tool to address the early recognition problem. To this last end, we introduce the use of the total gain and partial total gain to quantify recognition and early recognition of active compounds attributed to the variations of the scores obtained with virtual screening methods. Additionally to its usefulness in the evaluation of virtual screening methods, predictiveness curves can be used to define optimal score thresholds for the selection of compounds to be tested experimentally in a drug discovery program. We illustrate the use of predictiveness curves as a complement to ROC on the results of a virtual screening of the Directory of Useful Decoys datasets using three different methods (Surflex-dock, ICM, Autodock Vina).
  
  
  CONCLUSIONS
  The predictiveness curves cover different aspects of the predictive power of the scores, allowing a detailed evaluation of the performance of virtual screening methods. We believe predictiveness curves efficiently complete the set of tools available for the analysis of virtual screening results.
authors.affiliation: Laboratoire Génomique Bioinformatique et Applications, EA 4627, Conservatoire National des Arts et Métiers, 292 rue Saint Martin, 75003 Paris, France.
authors.forename: Charly
authors.initials: C
authors.lastname: Empereur-Mot
authors.affiliation: Laboratoire Génomique Bioinformatique et Applications, EA 4627, Conservatoire National des Arts et Métiers, 292 rue Saint Martin, 75003 Paris, France.
authors.forename: Hélène
authors.initials: H
authors.lastname: Guillemain
authors.affiliation: Equipe MSDMA, Laboratoire CEDRIC, EA 4629, Conservatoire National des Arts et Métiers, 292 rue Saint Martin, 75003 Paris, France.
authors.forename: Aurélien
authors.initials: A
authors.lastname: Latouche
authors.affiliation: Laboratoire Génomique Bioinformatique et Applications, EA 4627, Conservatoire National des Arts et Métiers, 292 rue Saint Martin, 75003 Paris, France.
authors.forename: Jean-François
authors.initials: JF
authors.lastname: Zagury
authors.affiliation: Université de Lyon, 69622 Lyon, France ; UMRESTTE, Université Lyon 1, 69373 Lyon, France ; UMRESTTE, IFSTTAR, 69675 Bron, France.
authors.forename: Vivian
authors.initials: V
authors.lastname: Viallon
authors.affiliation: Laboratoire Génomique Bioinformatique et Applications, EA 4627, Conservatoire National des Arts et Métiers, 292 rue Saint Martin, 75003 Paris, France.
authors.forename: Matthieu
authors.identifier: 0000-0001-5921-460X
authors.initials: M
authors.lastname: Montes