title: FQC: A novel approach for efficient compression, archival, and dissemination of fastq datasets.
abstract: Sequence data repositories archive and disseminate fastq data in compressed format. In spite of having relatively lower compression efficiency, data repositories continue to prefer GZIP over available specialized fastq compression algorithms. Ease of deployment, high processing speed and portability are the reasons for this preference. This study presents FQC, a fastq compression method that, in addition to providing significantly higher compression gains over GZIP, incorporates features necessary for universal adoption by data repositories/end-users. This study also proposes a novel archival strategy which allows sequence repositories to simultaneously store and disseminate lossless as well as (multiple) lossy variants of fastq files, without necessitating any additional storage requirements. For academic users, Linux, Windows, and Mac implementations (both 32 and 64-bit) of FQC are freely available for download at: https://metagenomics.atc.tcs.com/compression/FQC .
authors: Bio-Sciences R&D Division, TCS Innovation Labs, Tata Consultancy Services Limited, 54-B, Hadapsar Industrial Estate, Pune 411013, Maharashtra, India.
authors: Anirban
authors: A
authors: Dutta
authors: Mohammed Monzoorul
authors: MM
authors: Haque
authors: Tungadri
authors: T
authors: Bose
authors: C V S K
authors: CV
authors: Reddy
authors: Sharmila S
authors: SS
authors: Mande
mesh_terms: D000465:Algorithms; D044962:Data Compression / Q000379:methods*; D066289:Data Curation / Q000379:methods*; D008969:Molecular Sequence Data; D017422:Sequence Analysis, DNA / Q000379:methods*; D012984:Software
keywords: Data compaction and compression; NGS data; algorithms for biological data management; sequencing data archival