CHT4CPG: a hashing algorithm tailored for psuedo-genomes that effectively compresses FASTQ datasets

Motivation: High-throughput sequencing experiments can generate massive amounts of data in the form of short reads, which are subsequences of genomic sequences. A single experiment on the human genome can produce billions of such reads (with lengths around 100). Currently, the main DNA and RNA sequence databases store a large amount of DNA and RNA sequence information, and the scale of these sequence databases is growing rapidly. Due to the enormous size of the files involved, the compression of FASTQ files is crucial for the storage and transmission of genomic data, and to store and transmit the massive amounts of FASTQ data, efficient data compression algorithms must be utilized. Results: We present CHT4CPG (Constructing a Hash Table for Compressing PG), an algorithmic advancement over PgRC. This method is efficient for compressing DNA streams and is based on the concept of constructing a hash table to identify redundancy and encoding the redundancies to improve compression ratio. Our experiments demonstrate that CHT4CPG outperforms PgRC, its main competitor, by approximately 1-4% on average