LW-FQZip 2: a parallelized reference-based compression of FASTQ files.
BACKGROUND
The rapid progress of high-throughput DNA sequencing techniques has dramatically reduced the costs of whole genome sequencing, which leads to revolutionary advances in gene industry. The explosively increasing volume of raw data outpaces the decreasing disk cost and the storage of huge sequencing data has become a bottleneck of downstream analyses. Data compression is considered as a solution to reduce the dependency on storage. Efficient sequencing data compression methods are highly demanded.


RESULTS
In this article, we present a lossless reference-based compression method namely LW-FQZip 2 targeted at FASTQ files. LW-FQZip 2 is improved from LW-FQZip 1 by introducing more efficient coding scheme and parallelism. Particularly, LW-FQZip 2 is equipped with a light-weight mapping model, bitwise prediction by partial matching model, arithmetic coding, and multi-threading parallelism. LW-FQZip 2 is evaluated on both short-read and long-read data generated from various sequencing platforms. The experimental results show that LW-FQZip 2 is able to obtain promising compression ratios at reasonable time and memory space costs.


CONCLUSIONS
The competence enables LW-FQZip 2 to serve as a candidate tool for archival or space-sensitive applications of high-throughput DNA sequencing data. LW-FQZip 2 is freely available at http://csse.szu.edu.cn/staff/zhuzx/LWFQZip2 and https://github.com/Zhuzxlab/LW-FQZip2 .
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China.
Zhi-An
ZA
Huang
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China.
Zhenkun
Z
Wen
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China.
Qingjin
Q
Deng
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China.
Ying
Y
Chu
School of Medicine, Shenzhen University, Shenzhen, 518060, China.
Yiwen
Y
Sun
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China. zhuzx@szu.edu.cn.
Zexuan
Z
Zhu
D044962:Data Compression / Q000379:methods*; D059014:High-Throughput Nucleotide Sequencing / Q000379:methods*; D016415:Sequence Alignment / Q000379:methods*; D017422:Sequence Analysis, DNA / Q000379:methods*
High-throughput sequencing; Reference- based compression; Sequence alignment; Sequencing data compression