A Unified Model for Summarizing and Querying Multi-Format Content in English using Natural Language Processing

This paper presents a tool for summarizing content from various formats—text, PDFs, audio, video, YouTube videos, and images—into English, regardless of the original language. Current models like Bidirectional and Auto-Regressive Transformer (BART), Multilingual Bidirectional and Auto-Regressive Transformer (mBART), Web-based Highly Intelligent Speech Processing for Efficient Recognition (Whisper), LaMini-FLAN-v5-248M, and Easy Optical Character Recognition (EasyOCR) focus on specific tasks such as text summarization or audio transcription, but none offer a unified solution across multiple formats. This tool works by integrating these fine-tuned models to provide summarized content in English. Despite integration challenges, we developed a solution for multiple formats and languages. Additionally, a prompt-based search was developed to allow users to ask questions based on the generated summaries. This tool is proposed to address the challenges faced by students, researchers, and professionals such as journalists, business analysts, and educators in processing large amounts of information in diverse formats efficiently. Experimental results show strong performance for text summarization, with ROUGE-1: 48.21%, ROUGE-2: 35.31%, and ROUGE-L: 45.67%. Also, we observed 96% accuracy for language translation. Finally, for image-text extraction, we found a Word Error Rate (WER) of 29.26% and a Character Error Rate (CER) of 9.20%, a WER of 25.78% and a CER of 5.33% for audio-text extraction and a WER of 20.25% and a CER of 9.93% for document-text extraction.