MalFormer: A Novel Vision Transformer Model for Robust Malware Analysis

As traditional signature-based malware analysis struggles to detect malware variants, image-based malware analysis has been researched to overcome these limitations. Especially, Vision Transformer (ViT)-based malware analysis has achieved high performance by capturing global features of malware images. However, previous ViT-based methods struggle to effectively capture both local and global features of images and exhibit limited generalization performance. In this paper, we propose a novel malware analysis method that introduces MalFormer, which employs the Cross-Scale Embedding Layer, Long-Short Distance Attention, and Adaptive Token Masking. Also, we introduce Sharpness Aware Minimization (SAM) to optimize the training process of MalFormer. The proposed method enhances sensitivity to key features while maintaining a well-balanced integration of local and global features. From the experimental results on various datasets, we show that the proposed method outperforms state-of-the-art ViT-based methods.We also show that the proposed method provides strong robustness against obfuscation techniques, achieving high performance on obfuscated samples.