Enhancing Abstractive Summarization with T5SAM: A Transformer-Based Approach

Abstractive text summarization (ATS) tries to produce short summaries by paraphrasing the input text in a way that retains its underlying meaning. As the demand for automatic summarization has increased across domains, the Transformer-based models have become the state of the art models of the present time. In this paper, a T5SAM model, T5 as a transformer is proposed for abstractive text summarization on a blend of Inshorts News and DUC-2004 datasets. The approach here is to boost the effectiveness of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{T 5}$</tex> model using a combination of incline accumulation, mixed precision (fp16), and controlled encoder freezing in an attempt to optimize maximum model efficiency and generalization. The performance of the model is measured according to ROUGE metrics, with ROUGE 1: 59.58, ROUGE 2: 36.08, and ROUGE L: 59.20, performing much better than the existing baseline models. The key improvements are in the reduction of exposure bias, improved content retention, and fluency and coherence of output summaries. These results confirm the efficacy of optimizing the pre-trained Transformer models to solve domain-specific summarization tasks and come up with a scalable and robust solution for real-world applications.