title: Methods to increase reproducibility in differential gene expression via meta-analysis.
abstract: Findings from clinical and biological studies are often not reproducible when tested in independent cohorts. Due to the testing of a large number of hypotheses and relatively small sample sizes, results from whole-genome expression studies in particular are often not reproducible. Compared to single-study analysis, gene expression meta-analysis can improve reproducibility by integrating data from multiple studies. However, there are multiple choices in designing and carrying out a meta-analysis. Yet, clear guidelines on best practices are scarce. Here, we hypothesized that studying subsets of very large meta-analyses would allow for systematic identification of best practices to improve reproducibility. We therefore constructed three very large gene expression meta-analyses from clinical samples, and then examined meta-analyses of subsets of the datasets (all combinations of datasets with up to N/2 samples and K/2 datasets) compared to a 'silver standard' of differentially expressed genes found in the entire cohort. We tested three random-effects meta-analysis models using this procedure. We showed relatively greater reproducibility with more-stringent effect size thresholds with relaxed significance thresholds; relatively lower reproducibility when imposing extraneous constraints on residual heterogeneity; and an underestimation of actual false positive rate by Benjamini-Hochberg correction. In addition, multivariate regression showed that the accuracy of a meta-analysis increased significantly with more included datasets even when controlling for sample size.
authors.affiliation: Stanford Institute for Immunity, Transplantation and Infection, Stanford University School of Medicine, Stanford, CA 94305, USA.|Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Timothy E
authors.identifier: 0000-0002-3596-1093
authors.initials: TE
authors.lastname: Sweeney
authors.affiliation: Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Winston A
authors.initials: WA
authors.lastname: Haynes
authors.affiliation: Stanford Institute for Immunity, Transplantation and Infection, Stanford University School of Medicine, Stanford, CA 94305, USA.|Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Francesco
authors.initials: F
authors.lastname: Vallania
authors.affiliation: Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA 94305, USA.|Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA 94305, USA.|Meta-research Innovation Center at Stanford (METRICS), Stanford, CA 94305, USA.
authors.forename: John P
authors.initials: JP
authors.lastname: Ioannidis
authors.affiliation: Stanford Institute for Immunity, Transplantation and Infection, Stanford University School of Medicine, Stanford, CA 94305, USA pkhatri@stanford.edu.|Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Purvesh
authors.initials: P
authors.lastname: Khatri
mesh_terms: D000230:Adenocarcinoma / Q000235:genetics / Q000473:pathology; D000077192:Adenocarcinoma of Lung; D009202:Cardiomyopathies / Q000235:genetics / Q000473:pathology; D015331:Cohort Studies; D066264:Datasets as Topic; D020869:Gene Expression Profiling; D005786:Gene Expression Regulation; D018740:Genetic Heterogeneity; D015894:Genome, Human; D055106:Genome-Wide Association Study / Q000706:statistics & numerical data*; D006084:Graft Rejection / Q000235:genetics / Q000473:pathology; D017408:Guidelines as Topic; D006801:Humans; D007251:Influenza, Human / Q000235:genetics / Q000473:pathology; D016030:Kidney Transplantation; D008175:Lung Neoplasms / Q000235:genetics / Q000473:pathology; D015201:Meta-Analysis as Topic; D015233:Models, Statistical; D015203:Reproducibility of Results; D018401:Sample Size; D018805:Sepsis / Q000235:genetics / Q000473:pathology; D014397:Tuberculosis, Pulmonary / Q000235:genetics / Q000473:pathology