Comparative Analysis of Text Classification Techniques for Phishing and Smishing Detection Using Term Frequency–Inverse Document Frequency and Gemini Sentence Embeddings

This study presents an approach to phishing and smishing detection using text classification algorithms and two feature extraction techniques: term frequency–inverse document frequency (TF-IDF) and Gemini sentence embeddings. The research uses a curated, self-collected dataset of 27,000 messages, including 2,000 synthetic records to increase linguistic diversity. With TF-IDF, the support vector classifier (SVC) achieved a training accuracy of 97.85% and a test accuracy of 89.37%, indicating a generalization gap. Multilayer perceptron (MLP) and stochastic gradient descent classifiers achieved test accuracies of 88.28% and 86.06%, respectively. Sentence embeddings showed stronger performance, with SVC achieving 94.69% test accuracy, followed by MLP at 92.70% and K-nearest neighbors (KNN) at 92.35%. The decision tree classifier showed comparatively lower performance, with test accuracies of 78.04% using TF-IDF and 64.04% using sentence embeddings. Overall, sentence embeddings outperformed TF-IDF in the evaluated experimental setting. Unlike studies evaluating traditional features or semantic embeddings separately, this work provides a controlled comparison across feature representations, classifiers, and communication channels (email and SMS). The study examines generalization behavior, model stability, and feature representation effects beyond peak accuracy, contributing a structured evaluation framework for natural language processing-based cybersecurity research.