On the feasibility of using an encoder-only model for anomaly detection: the BERTAD approach

The Internet of Things has provided the necessary infrastructure for connected devices to collaborate and share information intelligently. Detecting anomalies in the traffic generated by the ’things’ is pivotal to providing cybersecurity in the Edge-Cloud continuum since vulnerabilities and security risks increase as more and more devices are integrated into daily life. Recent approaches perform anomalous IoT traffic detection using diverse machine-learning techniques, but they do not leverage the capability provided by generative artificial intelligence - particularly large language models - to analyze high-dimensional data. This paper presents an approach called BERTAD to investigate the feasibility of using the encoder-only architecture to detect anomalous traffic in IoT. BERTAD includes a model based on the Bidirectional Encoder Representations from Transformers for detecting anomalies in IoT. Also, it encompasses a benchmark that compares BERTAD to several traditional machine learning and deep learning models. Results revealed that BERTAD is a promising solution for facing anomalies in IoT traffic. BERTAD, an encoder-only solution, achieved outstanding Precision and Accuracy and good Matthew’s Correlation Coefficient, corroborating its capability to detect anomalies appropriately. Those results were higher or similar to those obtained by Recurrent and Convolutional Neural Networks, Random Forest, and Extreme Gradient Boosting machine learning techniques.