title: Experiences Building Globus Genomics: A Next-Generation Sequencing Analysis Service using Galaxy, Globus, and Amazon Web Services.
abstract: We describe Globus Genomics, a system that we have developed for rapid analysis of large quantities of next-generation sequencing (NGS) genomic data. This system achieves a high degree of end-to-end automation that encompasses every stage of data analysis including initial data retrieval from remote sequencing centers or storage (via the Globus file transfer system); specification, configuration, and reuse of multi-step processing pipelines (via the Galaxy workflow system); creation of custom Amazon Machine Images and on-demand resource acquisition via a specialized elastic provisioner (on Amazon EC2); and efficient scheduling of these pipelines over many processors (via the HTCondor scheduler). The system allows biomedical researchers to perform rapid analysis of large NGS datasets in a fully automated manner, without software installation or a need for any local computing infrastructure. We report performance and cost results for some representative workloads.
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Ravi K
authors.initials: RK
authors.lastname: Madduri
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Dinanath
authors.initials: D
authors.lastname: Sulakhe
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Lukasz
authors.initials: L
authors.lastname: Lacinski
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Bo
authors.initials: B
authors.lastname: Liu
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Alex
authors.initials: A
authors.lastname: Rodriguez
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Kyle
authors.initials: K
authors.lastname: Chard
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Utpal J
authors.initials: UJ
authors.lastname: Dave
authors.affiliation: Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL.
authors.forename: Ian T
authors.initials: IT
authors.lastname: Foster
keywords: Cloud; HPC; HTC; NGS; workflows