Performance Analysis of Transformer Models in Text Summarization with Insights for Future Ensemble Techniques

This paper explores the capabilities of transformer-based models BART, RoBERTa, and PEGASUS in abstractive text summarization. It uses a large diverse dataset of 200,000 rows, which was found to illustrate their performance in different content types, including conversational, journalistic, and transcribed texts. Preprocessing techniques used for model training included tokenization, lemmatization, and vectorization. Evaluation metrics BLEU and ROUGE were used to assess the quality of the output. A stack-based ensemble approach is proposed as a future enhancement, making use of the strengths of individual models to optimize coherence, accuracy, and generalization. Deployment considerations including scalability, latency, and cloud-based hosting are discussed with practical strategies for real-world implementation.