Comparative Analysis of LLM-Generated Summaries using Traditional Extractive Methods: Algorithmic Fidelity and Performance Evaluation

This study investigates the potential of large language models (LLMs) to replicate traditional extractive summarization algorithms such as TextRank, TF-IDF, LSA, Luhn’s method, and SumBasic. We evaluate three LLMs—Llama, Qwen, and Gemma—on 1,000 CNN/Daily Mail articles. The summaries are compared with those from classical methods using lexical (ROUGE, BLEU), semantic (BERT and TF-IDF cosine similarity), and compression ratio metrics. Additionally, we introduce a novel metric for algorithmic compliance, measuring exact sentence matches. Results show LLM-generated summaries outperform traditional methods in semantic alignment and lexical overlap, with compliance rates ranging from 65% to 72%. Llama achieves the highest compliance and overlap, while Gemma offers stable performance and Qwen shows more variability. These findings suggest that LLMs, when properly prompted, can match traditional methods while offering more flexibility, highlighting the need for hybrid evaluation frameworks.