title: Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures.
abstract: There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard 'dashboard' of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols.
authors.affiliation: 1] National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland 20899, USA [2] Department of Bioengineering, Stanford University, 443 Via Ortega, Stanford, California 94305, USA.
authors.forename: Sarah A
authors.initials: SA
authors.lastname: Munro
authors.affiliation: National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland 20899, USA.
authors.forename: Steven P
authors.initials: SP
authors.lastname: Lund
authors.affiliation: 1] National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland 20899, USA [2] Department of Bioengineering, Stanford University, 443 Via Ortega, Stanford, California 94305, USA.
authors.forename: P Scott
authors.initials: PS
authors.lastname: Pine
authors.affiliation: Interdisciplinary Centre for Bioinformatics, University of Leipzig, Härtelstrasse 16-18, 04107 Leipzig, Germany.
authors.forename: Hans
authors.initials: H
authors.lastname: Binder
authors.affiliation: Institute of Bioinformatics, Johannes Kepler University, Altenberger Str. 69, 4040 Linz, Austria.
authors.forename: Djork-Arné
authors.initials: DA
authors.lastname: Clevert
authors.affiliation: Computational Genomics Program, Principe Felipe Research Center, Avd Eduardo Primo Yúfera 3, 46012 Valencia, Spain.
authors.forename: Ana
authors.initials: A
authors.lastname: Conesa
authors.affiliation: 1] Computational Genomics Program, Principe Felipe Research Center, Avd Eduardo Primo Yúfera 3, 46012 Valencia, Spain [2] CIBER de Enfermedades Raras (CIBERER) and Functional Genomics Node, INB., Valencia, Spain.
authors.forename: Joaquin
authors.initials: J
authors.lastname: Dopazo
authors.affiliation: ecSeq Bioinformatics, Brandvorwerkstrasse 43, 04275 Leipzig, Germany.
authors.forename: Mario
authors.initials: M
authors.lastname: Fasold
authors.affiliation: Institute of Bioinformatics, Johannes Kepler University, Altenberger Str. 69, 4040 Linz, Austria.
authors.forename: Sepp
authors.initials: S
authors.lastname: Hochreiter
authors.affiliation: National Center for Toxicological Research, Food and Drug Administration, 3900 NCTR Road, Jefferson, Arkansas 72079, USA.
authors.forename: Huixiao
authors.initials: H
authors.lastname: Hong
authors.affiliation: Genomics Core Facility, Feinberg School of Medicine, Northwestern University, Tarry building 2-757, 300 E. Superior St. Chicago, Illinois 60611, USA.
authors.forename: Nadereh
authors.initials: N
authors.lastname: Jafari
authors.affiliation: 1] Chair of Bioinformatics, Boku University Vienna, Muthgasse 18, Vienna 1190, Austria [2] University of Warwick, Coventry CV4 7AL, UK.
authors.forename: David P
authors.initials: DP
authors.lastname: Kreil
authors.affiliation: Chair of Bioinformatics, Boku University Vienna, Muthgasse 18, Vienna 1190, Austria.
authors.forename: Paweł P
authors.initials: PP
authors.lastname: Łabaj
authors.affiliation: Department of Physiology and Biophysics, Institute for Computational Biomedicine, Weill Cornell Medical College, 1305 York Avenue, Room Y13-04, Box 140, New York, New York 10021, USA.
authors.forename: Sheng
authors.initials: S
authors.lastname: Li
authors.affiliation: 1] Division of Bioinformatics, The Walter and Eliza Hall Institute of Medical Research, 1G Royal Parade, Parkville, Victoria 3052, Australia [2] Department of Medical Biology, The University of Melbourne, Parkville, Victoria 3010, Australia.
authors.forename: Yang
authors.initials: Y
authors.lastname: Liao
authors.affiliation: Nationwide Children's Hospital, Columbus, Ohio 43205, USA.
authors.forename: Simon M
authors.initials: SM
authors.lastname: Lin
authors.affiliation: National Center for Toxicological Research, Food and Drug Administration, 3900 NCTR Road, Jefferson, Arkansas 72079, USA.
authors.forename: Joseph
authors.initials: J
authors.lastname: Meehan
authors.affiliation: Department of Physiology and Biophysics, Institute for Computational Biomedicine, Weill Cornell Medical College, 1305 York Avenue, Room Y13-04, Box 140, New York, New York 10021, USA.
authors.forename: Christopher E
authors.initials: CE
authors.lastname: Mason
authors.affiliation: 1] CIBER de Enfermedades Raras (CIBERER) and Functional Genomics Node, INB., Valencia, Spain [2] Medical Genome Project, Genomics and Bioinformatics Platform of Andalusia, c/ Albert Einstein s/n, 41092 Sevilla, Spain.
authors.forename: Javier
authors.initials: J
authors.lastname: Santoyo-Lopez
authors.affiliation: Thermo Fisher Scientific, Research &Development, 2170 Woodward Street, Austin, Texas 78744, USA.
authors.forename: Robert A
authors.initials: RA
authors.lastname: Setterquist
authors.affiliation: State Key Laboratory of Genetic Engineering and MOE Key Laboratory of Contemporary Anthropology, Schools of Life Sciences and Pharmacy, Fudan University, Shanghai 201203, China.
authors.forename: Leming
authors.initials: L
authors.lastname: Shi
authors.affiliation: 1] Division of Bioinformatics, The Walter and Eliza Hall Institute of Medical Research, 1G Royal Parade, Parkville, Victoria 3052, Australia [2] Department of Computing and Information Systems, The University of Melbourne, Parkville, Victoria 3010, Australia.
authors.forename: Wei
authors.initials: W
authors.lastname: Shi
authors.affiliation: 1] Division of Bioinformatics, The Walter and Eliza Hall Institute of Medical Research, 1G Royal Parade, Parkville, Victoria 3052, Australia [2] Department of Mathematics and Statistics, The University of Melbourne, Parkville, Victoria 3010, Australia.
authors.forename: Gordon K
authors.initials: GK
authors.lastname: Smyth
authors.affiliation: Chair of Bioinformatics, Boku University Vienna, Muthgasse 18, Vienna 1190, Austria.
authors.forename: Nancy
authors.initials: N
authors.lastname: Stralis-Pavese
authors.affiliation: National Center for Toxicological Research, Food and Drug Administration, 3900 NCTR Road, Jefferson, Arkansas 72079, USA.
authors.forename: Zhenqiang
authors.initials: Z
authors.lastname: Su
authors.affiliation: National Center for Toxicological Research, Food and Drug Administration, 3900 NCTR Road, Jefferson, Arkansas 72079, USA.
authors.forename: Weida
authors.initials: W
authors.lastname: Tong
authors.affiliation: Division of Microbiology and Molecular Genetics, Center for Genomics, School of Medicine, Loma Linda University, Loma Linda, California 92350, USA.
authors.forename: Charles
authors.initials: C
authors.lastname: Wang
authors.affiliation: Research Informatics, Eli Lilly and Company, Lilly Corporate Center, Indianapolis, Indiana 46285, USA.
authors.forename: Jian
authors.initials: J
authors.lastname: Wang
authors.affiliation: National Center for Toxicological Research, Food and Drug Administration, 3900 NCTR Road, Jefferson, Arkansas 72079, USA.
authors.forename: Joshua
authors.initials: J
authors.lastname: Xu
authors.affiliation: Biomedical Informatics Research Center, Marshfield Clinic Research Foundation, 1000 N Oak Avenue, Marshfield, Wisconsin 54449, USA.
authors.forename: Zhan
authors.initials: Z
authors.lastname: Ye
authors.affiliation: Research Informatics, Eli Lilly and Company, Lilly Corporate Center, Indianapolis, Indiana 46285, USA.
authors.forename: Yong
authors.initials: Y
authors.lastname: Yang
authors.affiliation: State Key Laboratory of Genetic Engineering and MOE Key Laboratory of Contemporary Anthropology, Schools of Life Sciences and Pharmacy, Fudan University, Shanghai 201203, China.
authors.forename: Ying
authors.initials: Y
authors.lastname: Yu
authors.affiliation: 1] National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, Maryland 20899, USA [2] Department of Bioengineering, Stanford University, 443 Via Ortega, Stanford, California 94305, USA.
authors.forename: Marc
authors.initials: M
authors.lastname: Salit
mesh_terms: D020869:Gene Expression Profiling / Q000379:methods* / Q000592:standards; D006801:Humans; D012333:RNA, Messenger / Q000235:genetics*; D012015:Reference Standards; D015203:Reproducibility of Results
chemical_list: D012333:RNA, Messenger