title: Deciding when to stop: efficient experimentation to learn to predict drug-target interactions.
abstract: BACKGROUND
  Active learning is a powerful tool for guiding an experimentation process. Instead of doing all possible experiments in a given domain, active learning can be used to pick the experiments that will add the most knowledge to the current model. Especially, for drug discovery and development, active learning has been shown to reduce the number of experiments needed to obtain high-confidence predictions. However, in practice, it is crucial to have a method to evaluate the quality of the current predictions and decide when to stop the experimentation process. Only by applying reliable stopping criteria to active learning can time and costs in the experimental process actually be saved.
  
  
  RESULTS
  We compute active learning traces on simulated drug-target matrices in order to determine a regression model for the accuracy of the active learner. By analyzing the performance of the regression model on simulated data, we design stopping criteria for previously unseen experimental matrices. We demonstrate on four previously characterized drug effect data sets that applying the stopping criteria can result in upto 40 % savings of the total experiments for highly accurate predictions.
  
  
  CONCLUSIONS
  We show that active learning accuracy can be predicted using simulated data and results in substantial savings in the number of experiments required to make accurate drug-target predictions.
authors.affiliation: Freiburg Institute for Advanced Studies, University of Freiburg, Freiburg, Germany. temerina@frias.uni-freiburg.de.
authors.forename: Maja
authors.initials: M
authors.lastname: Temerinac-Ott
authors.affiliation: Computational Biology Department, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, 15213, PA, USA. rumi@cmu.edu.
authors.forename: Armaghan W
authors.initials: AW
authors.lastname: Naik
authors.affiliation: Freiburg Institute for Advanced Studies, University of Freiburg, Freiburg, Germany. murphy@cmu.edu.|Computational Biology Department, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, 15213, PA, USA. murphy@cmu.edu.|Departments of Biological Sciences, Biomedical Engineering and Machine Learning, Carnegie Mellon University, 5000 Forbes Ave15213, Pittsburgh, PA, USA. murphy@cmu.edu.
authors.forename: Robert F
authors.initials: RF
authors.lastname: Murphy
mesh_terms: D000465:Algorithms; D055808:Drug Discovery; D006801:Humans; D008954:Models, Biological; D015233:Models, Statistical; D004364:Pharmaceutical Preparations / Q000378:metabolism*; D011506:Proteins / Q000378:metabolism*; D012107:Research Design
chemical_list: D004364:Pharmaceutical Preparations; D011506:Proteins