AFastQF: An Automated Fastq Quality Filtering of High-Throughput Genomic Data

Next-Generation Sequencing has revolutionized bioinformatics research by enabling high-throughput sequencing of DNA and RNA.However, the exponential growth in genomic data presents challenges in managing and analyzing large datasets.This study proposes an automated data filtering methodology to streamline NGS data preprocessing.Using an asortment of data structure algorithms and sliding window-based approach, the method systematically filters adapters, duplicates and low-quality reads from Fastq data, improving data quality and reducing manual curation efforts as well as storage requirement.The methodology introduces a duplicate removal technique at the preprocessing stage and managed to obtain high speed data filtering.Integrated into existing pipelines, the methodology ensures compatibility and scalability along with reduced processing time.The automated data filtering methodology addresses the critical need for efficient and reliable Next-Generation Sequencing data preprocessing, accelerating genomic research and enhancing biological discoveries.