title: MetaSpark: a spark-based distributed processing tool to recruit metagenomic reads to reference genomes.
abstract: With the advent of next-generation sequencing, traditional bioinformatics tools are challenged by massive raw metagenomic datasets. One of the bottlenecks of metagenomic studies is lack of large-scale and cloud computing suitable data analysis tools. In this paper, we proposed a Spark based tool, called MetaSpark, to recruit metagenomic reads to reference genomes. MetaSpark benefits from the distributed data set (RDD) of Spark, which makes it able to cache data set in memory across cluster nodes and scale well with the datasets. Compared with previous metagenomics recruitment tools, MetaSpark recruited significantly more reads than many programs such as SOAP2, BWA and LAST and increased recruited reads by ∼4% compared with FR-HIT when there were 1 million reads and 0.75 GB references. Different test cases demonstrate MetaSpark's scalability and overall high performance.
  
  
  
  https://github.com/zhouweiyg/metaspark.
  
  
  
  bniu@sccas.cn , jingluo@ynu.edu.cn.
  
  
  
  Supplementary data are available at Bioinformatics online.
authors.affiliation: School of Software, Yunnan University, Kunming, China.
authors.forename: Wei
authors.initials: W
authors.lastname: Zhou
authors.affiliation: Computer Network Information Center of Chinese Academy of Sciences, Beijing, China.|University of Chinese Academy of Sciences, Beijing 100190, China.
authors.forename: Ruilin
authors.initials: R
authors.lastname: Li
authors.affiliation: School of Software, Yunnan University, Kunming, China.
authors.forename: Shuo
authors.initials: S
authors.lastname: Yuan
authors.affiliation: School of Software, Yunnan University, Kunming, China.
authors.forename: ChangChun
authors.initials: C
authors.lastname: Liu
authors.affiliation: School of Software, Yunnan University, Kunming, China.
authors.forename: Shaowen
authors.initials: S
authors.lastname: Yao
authors.affiliation: School of Life Sciences and State Key Laboratory for Conservation and Utilization of Bio-Resources in Yunnan, Yunnan University, Kunming, China.
authors.forename: Jing
authors.initials: J
authors.lastname: Luo
authors.affiliation: Computer Network Information Center of Chinese Academy of Sciences, Beijing, China.|University of Chinese Academy of Sciences, Beijing 100190, China.
authors.forename: Beifang
authors.initials: B
authors.lastname: Niu
mesh_terms: D000465:Algorithms; D016678:Genome; D059014:High-Throughput Nucleotide Sequencing / Q000379:methods* / Q000592:standards; D006801:Humans; D056186:Metagenomics / Q000379:methods* / Q000592:standards; D012015:Reference Standards; D012984:Software