Enhancing Cybersecurity with Transformers: Preventing Phishing Emails and Social Media Scams

In this study, the enhancement of cybersecurity through the use of transformer models is explored, specifically focusing on the prevention of phishing emails and social media scams. Two modern datasets were utilized: the Phishing_Pot dataset, which provided phishing emails across 19 languages, and a collection of legitimate emails. Additionally, a dataset from Baaz, an Arabic social media platform, was compiled. Four approaches to phishing detection were compared: zero-shot learning with large language models (LLMs), transformers for feature extraction followed by random forest classification, fine-tuning of small language models (SLMs) on newly generated datasets, and an ensemble method combining the top models from the previous approaches. The experiments demonstrated that LLMs such as GPT-4o, Gemini 1.5, and Claude 3 achieved high accuracy, while SLMs showed improved performance post fine-tuning. The ensemble methods can be used to enhance the accuracy of fine-tuned SLM for phishing detection and scam prevention, reaching an accuracy of 98.50% for the first dataset and 95.25% for the second.