title: The Cancer Genomics Hub (CGHub): overcoming cancer through the power of torrential data.
abstract: The Cancer Genomics Hub (CGHub) is the online repository of the sequencing programs of the National Cancer Institute (NCI), including The Cancer Genomics Atlas (TCGA), the Cancer Cell Line Encyclopedia (CCLE) and the Therapeutically Applicable Research to Generate Effective Treatments (TARGET) projects, with data from 25 different types of cancer. The CGHub currently contains >1.4 PB of data, has grown at an average rate of 50 TB a month and serves >100 TB per week. The architecture of CGHub is designed to support bulk searching and downloading through a Web-accessible application programming interface, enforce patient genome confidentiality in data storage and transmission and optimize for efficiency in access and transfer. In this article, we describe the design of these three components, present performance results for our transfer protocol, GeneTorrent, and finally report on the growth of the system in terms of data stored and transferred, including estimated limits on the current architecture. Our experienced-based estimates suggest that centralizing storage and computational resources is more efficient than wide distribution across many satellite labs. Database URL: https://cghub.ucsc.edu.
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA cwilks@soe.ucsc.edu.
authors.forename: Christopher
authors.initials: C
authors.lastname: Wilks
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Melissa S
authors.initials: MS
authors.lastname: Cline
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Erich
authors.initials: E
authors.lastname: Weiler
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Mark
authors.initials: M
authors.lastname: Diehkans
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Craft
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Christy
authors.initials: C
authors.lastname: Martin
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: Murphy
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Howdy
authors.initials: H
authors.lastname: Pierce
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: John
authors.initials: J
authors.lastname: Black
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Donavan
authors.initials: D
authors.lastname: Nelson
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Litzinger
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Thomas
authors.initials: T
authors.lastname: Hatton
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Lori
authors.initials: L
authors.lastname: Maltbie
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Michael
authors.initials: M
authors.lastname: Ainsworth
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Patrick
authors.initials: P
authors.lastname: Allen
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Linda
authors.initials: L
authors.lastname: Rosewood
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Mitchell
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Bradley
authors.initials: B
authors.lastname: Smith
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Jim
authors.initials: J
authors.lastname: Warner
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: John
authors.initials: J
authors.lastname: Groboske
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Haifang
authors.initials: H
authors.lastname: Telc
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: Wilson
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Sanford
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Hannes
authors.initials: H
authors.lastname: Schmidt
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: David
authors.initials: D
authors.lastname: Haussler
authors.affiliation: Biomolecular Engineering, School of Engineering, University of California Santa Cruz, Santa Cruz, CA, USA, Center for Biomolecular Science and Engineering, School of Engineering, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA, Annai Systems Inc., 2100 Palomar Airport Road, Suite 210 Carlsbad, California 92011, USA, Cardinal Peak, LLC, 1380 Forest Park Circle, Suite 202 Lafayette, CO 80026, USA and Information Technology Services, University of California Santa Cruz (UCSC), Santa Cruz, CA 95064, USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: Maltbie
mesh_terms: D016494:Computer Security; D003628:Database Management Systems; D030541:Databases, Genetic; D057286:Electronic Health Records; D023281:Genomics / Q000379:methods*; D006801:Humans; D020407:Internet; D009369:Neoplasms / Q000235:genetics*