Using Hybrid Transformer and Convolutional Neural Network for Malware Detection in Internet of Things

Malicious firmware upgrading represents a critical security vulnerability in Internet of Things (IoT) devices. This study introduces HyCNNAt, a novel hybrid deep learning network for IoT malware detection that synergistically combines Convolutional Neural Networks (CNNs) with transformer attention mechanisms. HyCNNAt’s architecture vertically and horizontally stacks convolution and attention layers, enhancing the network’s generalization capabilities, capacity, and overall effectiveness. We evaluated HyCNNAt using a publicly available IoT firmware dataset, where it demonstrated superior performance with the highest accuracy ([Formula: see text]), F1-score ([Formula: see text]), and recall ([Formula: see text]), highlighting its robust classification capabilities, although its precision ([Formula: see text]) exhibited variability compared to state-of-the-art models such as CoAtNet, MobileViT, MobileNet, and MobileNet variants using transfer learning. These results underscore HyCNNAt’s potential as a robust solution for addressing the pressing challenge of IoT malware detection.