title: The Widening Gulf between Genomics Data Generation and Consumption: A Practical Guide to Big Data Transfer Technology.
abstract: In the last decade, high-throughput DNA sequencing has become a disruptive technology and pushed the life sciences into a distributed ecosystem of sequence data producers and consumers. Given the power of genomics and declining sequencing costs, biology is an emerging "Big Data" discipline that will soon enter the exabyte data range when all subdisciplines are combined. These datasets must be transferred across commercial and research networks in creative ways since sending data without thought can have serious consequences on data processing time frames. Thus, it is imperative that biologists, bioinformaticians, and information technology engineers recalibrate data processing paradigms to fit this emerging reality. This review attempts to provide a snapshot of Big Data transfer across networks, which is often overlooked by many biologists. Specifically, we discuss four key areas: 1) data transfer networks, protocols, and applications; 2) data transfer security including encryption, access, firewalls, and the Science DMZ; 3) data flow control with software-defined networking; and 4) data storage, staging, archiving and access. A primary intention of this article is to orient the biologist in key aspects of the data transfer process in order to frame their genomics-oriented needs to enterprise IT professionals.
authors.affiliation: Department of Genetics and Biochemistry, Clemson University, Clemson, SC, USA.
authors.forename: Frank A
authors.initials: FA
authors.lastname: Feltus
authors.affiliation: University of Utah Center for High Performance Computing, Salt Lake City, UT, USA.
authors.forename: Joseph R
authors.initials: JR
authors.lastname: Breen
authors.affiliation: Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA.
authors.forename: Juan
authors.initials: J
authors.lastname: Deng
authors.affiliation: Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA.
authors.forename: Ryan S
authors.initials: RS
authors.lastname: Izard
authors.affiliation: Clemson Computing and Information Technology, Clemson University, Anderson, SC, USA.
authors.forename: Christopher A
authors.initials: CA
authors.lastname: Konger
authors.affiliation: Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA.
authors.forename: Walter B
authors.initials: WB
authors.lastname: Ligon
authors.affiliation: NIH/NLM/NCBI, Bethesda, MD, USA.
authors.forename: Don
authors.initials: D
authors.lastname: Preuss
authors.affiliation: Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA.
authors.forename: Kuang-Ching
authors.initials: KC
authors.lastname: Wang
keywords: Big Data; data life cycle; networks; software-defined networking