title: On reliable discovery of molecular signatures.
abstract: BACKGROUND
  Molecular signatures are sets of genes, proteins, genetic variants or other variables that can be used as markers for a particular phenotype. Reliable signature discovery methods could yield valuable insight into cell biology and mechanisms of human disease. However, it is currently not clear how to control error rates such as the false discovery rate (FDR) in signature discovery. Moreover, signatures for cancer gene expression have been shown to be unstable, that is, difficult to replicate in independent studies, casting doubts on their reliability.
  
  
  RESULTS
  We demonstrate that with modern prediction methods, signatures that yield accurate predictions may still have a high FDR. Further, we show that even signatures with low FDR may fail to replicate in independent studies due to limited statistical power. Thus, neither stability nor predictive accuracy are relevant when FDR control is the primary goal. We therefore develop a general statistical hypothesis testing framework that for the first time provides FDR control for signature discovery. Our method is demonstrated to be correct in simulation studies. When applied to five cancer data sets, the method was able to discover molecular signatures with 5% FDR in three cases, while two data sets yielded no significant findings.
  
  
  CONCLUSIONS
  Our approach enables reliable discovery of molecular signatures from genome-wide data with current sample sizes. The statistical framework developed herein is potentially applicable to a wide range of prediction problems in bioinformatics.
authors.affiliation: Computational Biology, Department of Physics, Linköping University, SE58183 Linköping, Sweden. rnilsson@broad.mit.edu
authors.forename: Roland
authors.initials: R
authors.lastname: Nilsson
authors.forename: Johan
authors.initials: J
authors.lastname: Björkegren
authors.forename: Jesper
authors.initials: J
authors.lastname: Tegnér
mesh_terms: D000465:Algorithms; D014408:Biomarkers, Tumor / Q000235:genetics / Q000378:metabolism*; D019295:Computational Biology / Q000379:methods*; D003627:Data Interpretation, Statistical; D030541:Databases, Genetic; D005189:False Positive Reactions; D020869:Gene Expression Profiling / Q000379:methods / Q000706:statistics & numerical data; D020022:Genetic Predisposition to Disease / Q000235:genetics; D016678:Genome; D006801:Humans; D010641:Phenotype
chemical_list: D014408:Biomarkers, Tumor