title: Benchmarking substrate-based kinase activity inference using phosphoproteomic data.
abstract: BACKGROUND
  Phosphoproteomic experiments are increasingly used to study the changes in signaling occurring across different conditions. It has been proposed that changes in phosphorylation of kinase target sites can be used to infer when a kinase activity is under regulation. However, these approaches have not yet been benchmarked due to a lack of appropriate benchmarking strategies.
  
  
  RESULTS
  We used curated phosphoproteomic experiments and a gold standard dataset containing a total of 184 kinase-condition pairs where regulation is expected to occur to benchmark and compare different kinase activity inference strategies: Z-test, Kolmogorov Smirnov test, Wilcoxon rank sum test, gene set enrichment analysis (GSEA), and a multiple linear regression model. We also tested weighted variants of the Z-test and GSEA that include information on kinase sequence specificity as proxy for affinity. Finally, we tested how the number of known substrates and the type of evidence ( in vivo , in vitro or in silico ) supporting these influence the predictions.
  
  
  CONCLUSIONS
  Most models performed well with the Z-test and the GSEA performing best as determined by the area under the ROC curve (Mean AUC = 0.722). Weighting kinase targets by the kinase target sequence preference improves the results marginally. However, the number of known substrates and the evidence supporting the interactions has a strong effect on the predictions.
  
  
  METHODS
  The KSEA implementation is available in https://github.com/ evocellnet/ksea. Additional data is available in http://phosfate.com.
  
  
  BACKGROUND
  pbeltrao@ebi.ac.uk or ochoa@ebi.ac.uk.
  
  
  BACKGROUND
  Supplementary data are available at Bioinformatics online.
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors.forename: Claudia
authors.initials: C
authors.lastname: Hernandez-Armenta
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors.forename: David
authors.initials: D
authors.lastname: Ochoa
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors.forename: Emanuel
authors.initials: E
authors.lastname: Gonçalves
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.|RWTH Aachen University, Faculty of Medicine, Joint Research Center for Computational Biomedicine (JRC-COMBINE), Wendlingweg 2, Aachen, Germany.
authors.forename: Julio
authors.initials: J
authors.lastname: Saez-Rodriguez
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors.forename: Pedro
authors.initials: P
authors.lastname: Beltrao
mesh_terms: D003198:Computer Simulation; D006801:Humans; D010750:Phosphoproteins / Q000378:metabolism*; D010770:Phosphotransferases / Q000378:metabolism*; D040901:Proteomics / Q000379:methods*; D015398:Signal Transduction; D012984:Software
chemical_list: D010750:Phosphoproteins; D010770:Phosphotransferases