title: Measuring the wisdom of the crowds in network-based gene function inference.
abstract: BACKGROUND
  Network-based gene function inference methods have proliferated in recent years, but measurable progress remains elusive. We wished to better explore performance trends by controlling data and algorithm implementation, with a particular focus on the performance of aggregate predictions.
  
  
  RESULTS
  Hypothesizing that popular methods would perform well without hand-tuning, we used well-characterized algorithms to produce verifiably 'untweaked' results. We find that most state-of-the-art machine learning methods obtain 'gold standard' performance as measured in critical assessments in defined tasks. Across a broad range of tests, we see close alignment in algorithm performances after controlling for the underlying data being used. We find that algorithm aggregation provides only modest benefits, with a 17% increase in area under the ROC (AUROC) above the mean AUROC. In contrast, data aggregation gains are enormous with an 88% improvement in mean AUROC. Altogether, we find substantial evidence to support the view that additional algorithm development has little to offer for gene function prediction.
  
  
  METHODS
  The supplementary information contains a description of the algorithms, the network data parsed from different biological data resources and a guide to the source code (available at: http://gillislab.cshl.edu/supplements/).
authors.affiliation: Stanley Institute for Cognitive Genomics, Cold Spring Harbor Laboratory, Woodbury, NY 11797, USA.
authors.forename: W
authors.initials: W
authors.lastname: Verleyen
authors.affiliation: Stanley Institute for Cognitive Genomics, Cold Spring Harbor Laboratory, Woodbury, NY 11797, USA.
authors.forename: S
authors.initials: S
authors.lastname: Ballouz
authors.affiliation: Stanley Institute for Cognitive Genomics, Cold Spring Harbor Laboratory, Woodbury, NY 11797, USA.
authors.forename: J
authors.initials: J
authors.lastname: Gillis
mesh_terms: D000465:Algorithms; D019295:Computational Biology / Q000379:methods*; D030562:Databases, Protein; D053263:Gene Regulatory Networks; D006801:Humans; D053858:Metabolic Networks and Pathways / Q000502:physiology*; D008957:Models, Genetic; D011506:Proteins / Q000378:metabolism*; D012984:Software
chemical_list: D011506:Proteins