Evaluating the Text Summarization Efficiency of Large Language Models

Text summarization is crucial in condensing lengthy texts into concise summaries, enabling efficient information retrieval and content evaluation. Large language models (LLMs) have emerged as pivotal tools in this domain, demonstrating a remarkable ability to generate coherent summaries across diverse languages and domains. This study explores the effectiveness of LLMs, particularly Llama-2 and Gemma, in text summarization tasks. The study evaluates how well these LLMs autonomously generate summaries using zero-shot prompt engineering and supervised fine-tuning. Evaluation metrics such as BLEU, ROUGE, and BERTScore are utilized to evaluate the quality and coherence of the generated summaries. The study shows notable differences between Llama-2 and Gemma, with Llama-2 outperforming particularly well after fine-tuning. It achieves higher scores across all evaluation metrics, highlighting its ability to generate contextually relevant summaries more effectively than Gemma. Efforts to optimize Large Language Models (LLMs) for summarization should prioritize refining model architectures, exploring innovative fine-tuning methods, and expanding training datasets. These efforts are essential to fully harness the potential of LLMs in producing high-quality summaries across diverse domains and linguistic contexts.