TLECNN: A Transfer Learning-Enhanced CNN for Malware-Family Classification in IoT-Relevant Security Environment

The rapid growth of Internet of Things (IoT) devices has greatly expanded the attack surface for malware. It has created a need for accurate malware classification techniques suitable for resource-aware IoT security analysis in such environments. In this paper, a transfer learning-enhanced convolutional neural network (TLECNN) for image-based malware classification is proposed. This proposed framework is based on a ResNet50 backbone initialized with ImageNet pretrained weights and fine-tuned to classify malware samples from the MalImg dataset, containing 25 malware families. The framework systematically evaluates frozen, full, and selective fine-tuning strategies, including class-weighted learning, and retains selective fine-tuning without class weighting based on class-sensitive performance. The proposed model is evaluated using five optimization algorithms: Adam, NAdam, Adagrad, Adadelta, and Adamax. The experimental results show that the proposed TLECNN achieves an accuracy of 99.49%. In the reported optimizer evaluation, NAdam yielded the highest observed performance, with macro-precision of 98.87%, macro-recall of 98.73%, macro-F1-score of 98.79%, and MCC of 0.9940. The high macro-recall and F1-score are particularly relevant for malware family classification, where poor recognition of less frequent malware families may be concealed by high overall accuracy. The proposed framework demonstrates competitive performance in multiclass malware classification for centralized IoT-oriented security analysis. Computational characteristics were quantified, and the selected framework was further evaluated on an external multi-architecture ARM/MIPS and x86-64 ELF IoT malware dataset. These findings support the use of TLECNN for centralized or gateway-oriented IoT security analysis and provide a basis for extending transfer learning-based malware analysis to IoT-oriented cybersecurity applications.