title: Scaling up genome annotation using MAKER and work queue.
abstract: Next generation sequencing technologies have enabled sequencing many genomes. Because of the overall increasing demand and the inherent parallelism available in many required analyses, these bioinformatics applications should ideally run on clusters, clouds and/or grids. We present a modified annotation framework that achieves a speed-up of 45x using 50 workers using a Caenorhabditis japonica test case. We also evaluate these modifications within the Amazon EC2 cloud framework. The underlying genome annotation (MAKER) is parallelised as an MPI application. Our framework enables it to now run without MPI while utilising a wide variety of distributed computing resources. This parallel framework also allows easy explicit data transfer, which helps overcome a major limitation of bioinformatics tools that often rely on shared file systems. Combined, our proposed framework can be used, even during early stages of development, to easily run sequence analysis tools on clusters, grids and clouds.
authors.affiliation: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA.
authors.forename: Andrew
authors.initials: A
authors.lastname: Thrasher
authors.affiliation: Yelp, Inc., San Francisco, CA, USA.
authors.forename: Zachary
authors.initials: Z
authors.lastname: Musgrave
authors.affiliation: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Kachmarck
authors.affiliation: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA.
authors.forename: Douglas
authors.initials: D
authors.lastname: Thain
authors.affiliation: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USA.
authors.forename: Scott
authors.initials: S
authors.lastname: Emrich
mesh_terms: D000465:Algorithms; D000818:Animals; D000852:Anopheles / Q000235:genetics; D002107:Caenorhabditis / Q000235:genetics; D016000:Cluster Analysis; D019295:Computational Biology / Q000379:methods*; D003199:Computer Systems; D016678:Genome; D059014:High-Throughput Nucleotide Sequencing / Q000379:methods*; D012984:Software; D014370:Tsetse Flies / Q000235:genetics
keywords: Caenorhabditis japonica; bioinformatics; cloud computing; clusters; distributed computing; explicit data transfer; genome annotation; grid computing; next generation sequencing; work queue