High-Performance Abstractive Text Summarization Using Hierarchical Self-Attention and Distributed Pretrained Language Models

Introduction: Abstractive text summarization aims to generate concise and meaningful summaries by preserving salient information. However, existing machine learning, deep learning, and transformer-based models often suffer from factual inconsistency, hallucination, and high computational complexity, limiting their reliability. Materials and Methods: A Hierarchical Self-Attention–based Distributed Transformer enabled Bidirectional Pegasus LSTM (HSA-DistBPSTM) framework is proposed. The model employs hierarchical sparse self-attention for salient feature selection, a T5G descriptor for robust feature extraction, and a distributed learning strategy using multiple pretrained language models to enhance factual consistency and reduce overfitting. Results: The proposed framework demonstrates superior performance on the Hindi Text Short Summarization Corpus, achieving a BLEU_1 score of 0.95, CIDEr of 0.93, ROUGE-1 of 0.95, and ROUGE-2 of 0.93, indicating high-quality and reliable summary generation. Discussion: The hierarchical attention mechanism effectively reduces noise and hallucination by focusing on relevant contextual information, while distributed pretrained models improve robustness and scalability. The integration of T5G features further enhances generalization and summary coherence. conclusion: The HSA-DistBPSTM model provides an efficient and reliable solution for abstractive text summarization by addressing key challenges such as hallucination and factual inconsistency. Its strong performance validates its suitability for practical and multilingual NLP applications. Conclusion: The HSA-DistBPSTM model provides an efficient and reliable solution for abstractive text summarization by addressing key challenges such as hallucination and factual inconsistency. Its strong performance validates its suitability for practical and multilingual NLP applications.