Are large language models effective for detecting spam messages?

Abstract The widespread global use of short message service (SMS) has heightened the importance of reliable spam detection, as spam messages are not only disruptive but can also facilitate phishing and identity theft. While extensive prior research has explored SMS spam detection using traditional machine learning (ML), deep learning (DL), and transformer-based encoder models, the recent rapid advances in large language models (LLMs) with strong natural language understanding capabilities motivate a fresh, systematic evaluation. In this study, we investigate how effectively LLMs detect SMS spam using both English and Korean datasets. We benchmark representative prior approaches on recent SMS spam corpora and compare them against LLM-based classifiers. Our experiments cover both commercial black-box LLMs and locally deployable white-box models and examine multiple prompting strategies, including zero-shot, few-shot, and chain-of-thought (CoT) prompting. Furthermore, to reflect adversarial characteristics commonly used to evade detection, we evaluate robustness under concept drift and adversarial attack settings. Experimental results on English and Korean SMS datasets indicate that LLMs are highly robust for spam detection and can provide interpretable rationales by leveraging their language understanding capabilities, suggesting that LLMs are a highly effective and practical option for modern SMS spam filtering.