title: Comparative evaluation of gene set analysis approaches for RNA-Seq data.
abstract: BACKGROUND
  Over the last few years transcriptome sequencing (RNA-Seq) has almost completely taken over microarrays for high-throughput studies of gene expression. Currently, the most popular use of RNA-Seq is to identify genes which are differentially expressed between two or more conditions. Despite the importance of Gene Set Analysis (GSA) in the interpretation of the results from RNA-Seq experiments, the limitations of GSA methods developed for microarrays in the context of RNA-Seq data are not well understood.
  
  
  RESULTS
  We provide a thorough evaluation of popular multivariate and gene-level self-contained GSA approaches on simulated and real RNA-Seq data. The multivariate approach employs multivariate non-parametric tests combined with popular normalizations for RNA-Seq data. The gene-level approach utilizes univariate tests designed for the analysis of RNA-Seq data to find gene-specific P-values and combines them into a pathway P-value using classical statistical techniques. Our results demonstrate that the Type I error rate and the power of multivariate tests depend only on the test statistics and are insensitive to the different normalizations. In general standard multivariate GSA tests detect pathways that do not have any bias in terms of pathways size, percentage of differentially expressed genes, or average gene length in a pathway. In contrast the Type I error rate and the power of gene-level GSA tests are heavily affected by the methods for combining P-values, and all aforementioned biases are present in detected pathways.
  
  
  CONCLUSIONS
  Our result emphasizes the importance of using self-contained non-parametric multivariate tests for detecting differentially expressed pathways for RNA-Seq data and warns against applying gene-level GSA tests, especially because of their high level of Type I error rates for both, simulated and real data.
authors.affiliation: Division of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, 72205, USA. yrahmatallah@uams.edu.
authors.forename: Yasir
authors.initials: Y
authors.lastname: Rahmatallah
authors.affiliation: Computational Biology and Machine Learning Laboratory, Center for Cancer Research and Cell Biology, School of Medicine, Dentistry and Biomedical Sciences, Queen's University Belfast, 97 Lisburn Road, Belfast, BT9 7BL, UK. v@bio-complexity.com.
authors.forename: Frank
authors.initials: F
authors.lastname: Emmert-Streib
authors.affiliation: Division of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, 72205, USA. gvglazko@uams.edu.
authors.forename: Galina
authors.initials: G
authors.lastname: Glazko
mesh_terms: D003198:Computer Simulation; D005260:Female; D020869:Gene Expression Profiling / Q000379:methods*; D050172:Genes, X-Linked; D059014:High-Throughput Nucleotide Sequencing / Q000379:methods*; D006801:Humans; D008214:Lymphocytes / Q000378:metabolism; D008297:Male; D012313:RNA / Q000235:genetics*; D017423:Sequence Analysis, RNA / Q000379:methods*; D015398:Signal Transduction; D012984:Software
chemical_list: D012313:RNA