Lightweight Machine Learning Models for Mobile-Oriented SMS Spam Detection

Unsolicited and scam text messages are a compelling case for on-device filtering, but many recent detectors are based on deep or transformer models that may be large or slow enough to be challenging to run on phones. We evaluate eight classical classifiers—Multinomial and Complement Naive Bayes, Logistic Regression, a linear Support Vector Machine (SVM), a Decision Tree, a Random Forest, a small multi-layer perceptron and XGBoost—on the de-duplicated Short Message Service (SMS) Spam Collection (5,171 messages) from the University of California Irvine (UCI) repository using term frequency-inverse document frequency (TF-IDF) unigram and bigram features, tuned inside a nested ten-fold cross validation, comparing predictive quality, serialized model size and single-message inference latency. The linear SVM, Logistic Regression and Multinomial Naive Bayes form a top tier, with cross validated spam-class F1 scores of 0.951, 0.951 and 0.950 respectively—with no pairwise statistically significant differences among them—occupying 223–340 KiB and classifying a message in about 1 ms via a Python pipeline running on a desktop central processing unit (CPU). The Random Forest and the neural network made no accuracy improvements, while being roughly 13 and 18 times larger than the linear models. A study of vocabulary size shows that constraining the linear SVM to 2,000 terms reduces the model size to 88 KiB at a negligible sacrifice of 0.007 F1. A library-free export of Multinomial Naive Bayes, Logistic Regression and the linear SVM matched the scikit-learn predictions on all 1,035 held-out messages for possible Android implementation; on-device timings remain future work. Error analysis reveals that remaining errors tend to cluster in very short, conversational or announcement-like messages.