SMS spam detection using BERT and multi-graph convolutional networks

The surge in smartphone usage has significantly increased Short Message Service (SMS) traffic and, consequently, SMS spam, posing risks such as phishing, financial losses, and privacy breaches. Traditional rule-based and blacklist methods fail against evolving spamming techniques, prompting the adoption of machine learning and deep learning approaches. However, models like Convolutional Neural Networks and Recurrent Neural Networks struggle to capture global co-occurrence patterns and complex semantics, while transformer-based models like Bidirectional Encoder Representations from Transformers (BERT) lack explicit syntactic and co-occurrence modeling. To address these limitations, we propose the BERT with Triple-Graph Convolutional Networks (BERT-G3CN) model, the first framework to integrate BERT word em- beddings with graph embeddings from Co-occurrence, Heterogeneous, and Integrated Syntactic Graphs. This multigraph approach captures diverse features and models both global and local structures using tailored Graph Convolutional Networks. Experiments on two bench- mark datasets demonstrate that BERT-G3CN achieves superior accuracy of 99.28% and 93.78%, representing an improvement of approximately 2–3% over competitive baselines. • Propose BERT-G3CN, the first integration of BERT embeddings with triple-graph embeddings for SMS spam detection. • Capture global co-occurrence patterns, semantic relationships, and syntactic dependencies using Cooccurrence, Heterogeneous, and Integrated Syntactic Graphs. • Achieve superior performance with 99.28% and 93.78% accuracy on two benchmark datasets. • Demonstrate robustness in handling imbalanced and balanced SMS datasets with enhanced precision and recall. • Offer a novel approach to feature extraction for spam detection by combining graph embeddings with BERT’s contextual embeddings.