title: Benchmarking quantitative label-free LC-MS data processing workflows using a complex spiked proteomic standard dataset.
abstract: Proteomic workflows based on nanoLC-MS/MS data-dependent-acquisition analysis have progressed tremendously in recent years. High-resolution and fast sequencing instruments have enabled the use of label-free quantitative methods, based either on spectral counting or on MS signal analysis, which appear as an attractive way to analyze differential protein expression in complex biological samples. However, the computational processing of the data for label-free quantification still remains a challenge. Here, we used a proteomic standard composed of an equimolar mixture of 48 human proteins (Sigma UPS1) spiked at different concentrations into a background of yeast cell lysate to benchmark several label-free quantitative workflows, involving different software packages developed in recent years. This experimental design allowed to finely assess their performances in terms of sensitivity and false discovery rate, by measuring the number of true and false-positive (respectively UPS1 or yeast background proteins found as differential). The spiked standard dataset has been deposited to the ProteomeXchange repository with the identifier PXD001819 and can be used to benchmark other label-free workflows, adjust software parameter settings, improve algorithms for extraction of the quantitative metrics from raw MS data, or evaluate downstream statistical methods.
  
  
  CONCLUSIONS
  Bioinformatic pipelines for label-free quantitative analysis must be objectively evaluated in their ability to detect variant proteins with good sensitivity and low false discovery rate in large-scale proteomic studies. This can be done through the use of complex spiked samples, for which the "ground truth" of variant proteins is known, allowing a statistical evaluation of the performances of the data processing workflow. We provide here such a controlled standard dataset and used it to evaluate the performances of several label-free bioinformatics tools (including MaxQuant, Skyline, MFPaQ, IRMa-hEIDI and Scaffold) in different workflows, for detection of variant proteins with different absolute expression levels and fold change values. The dataset presented here can be useful for tuning software tool parameters, and also testing new algorithms for label-free quantitative analysis, or for evaluation of downstream statistical methods.
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: claire.ramus@cea.fr.
authors.forename: Claire
authors.initials: C
authors.lastname: Ramus
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: ahovasse@unistra.fr.
authors.forename: Agnès
authors.initials: A
authors.lastname: Hovasse
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: marlene.marcellin@ipbs.fr.
authors.forename: Marlène
authors.initials: M
authors.lastname: Marcellin
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: anne-marie.hesse@cea.fr.
authors.forename: Anne-Marie
authors.initials: AM
authors.lastname: Hesse
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: emmanuelle.mouton@ipbs.fr.
authors.forename: Emmanuelle
authors.initials: E
authors.lastname: Mouton-Barbosa
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: bouyssie@ipbs.fr.
authors.forename: David
authors.initials: D
authors.lastname: Bouyssié
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: sebastian.vaca@etu.unistra.fr.
authors.forename: Sebastian
authors.initials: S
authors.lastname: Vaca
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: ccarapito@unistra.fr.
authors.forename: Christine
authors.initials: C
authors.lastname: Carapito
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: karima.chaoui@ipbs.fr.
authors.forename: Karima
authors.initials: K
authors.lastname: Chaoui
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: christophe.bruley@cea.fr.
authors.forename: Christophe
authors.initials: C
authors.lastname: Bruley
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: jerome.garin@cea.fr.
authors.forename: Jérôme
authors.initials: J
authors.lastname: Garin
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: sarah.cianferani@unistra.fr.
authors.forename: Sarah
authors.initials: S
authors.lastname: Cianférani
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: myriam.ferro@cea.fr.
authors.forename: Myriam
authors.initials: M
authors.lastname: Ferro
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: vandors@unistra.fr.
authors.forename: Alain
authors.initials: A
authors.lastname: Van Dorssaeler
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: schiltz@ipbs.fr.
authors.forename: Odile
authors.initials: O
authors.lastname: Burlet-Schiltz
authors.affiliation: ProFi, Proteomic French Infrastructure, France; Laboratoire de Spectrométrie de Masse BioOrganique (LSMBO), IPHC, Université de Strasbourg, CNRS, UMR7178, 25 Rue Becquerel, 67087 Strasbourg, France. Electronic address: christine.schaeffer@unistra.fr.
authors.forename: Christine
authors.initials: C
authors.lastname: Schaeffer
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CEA, DSV, iRTSV, Laboratoire de Biologie à Grande Echelle, Grenoble F-38054, France; INSERM U1038, Grenoble F-38054, France; Université Grenoble, F-38054, France. Electronic address: yohann.coute@cea.fr.
authors.forename: Yohann
authors.initials: Y
authors.lastname: Couté
authors.affiliation: ProFi, Proteomic French Infrastructure, France; CNRS UMR5089 Institut de Pharmacologie et de Biologie Structurale, 205 route de Narbonne, 31077 Toulouse, France; Université de Toulouse, 118 route de Narbonne, 31062 Toulouse, France. Electronic address: gonzalez@ipbs.fr.
authors.forename: Anne
authors.initials: A
authors.lastname: Gonzalez de Peredo
mesh_terms: D019985:Benchmarking / Q000379:methods / Q000592:standards*; D002853:Chromatography, Liquid / Q000592:standards*; D013058:Mass Spectrometry / Q000592:standards*; D020543:Proteome / Q000032:analysis* / Q000592:standards*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software; D012986:Software Validation; D013194:Staining and Labeling; D057188:Workflow
chemical_list: D020543:Proteome
keywords: Computational proteomics; Label-free quantification; MS signal analysis; NanoLC–MS/MS; Proteomic standard; Spectral counting