Toward a Robust Detection of PowerShell Malware against Code Mixing and Obfuscation by Using Sentence Transformer and Similarity Learning

Embedded PowerShell commands or scripts are among the most popular malware payloads. For malware that prioritizes stealthiness, such as fileless malware, PowerShell’s access to Windows API functions without additional libraries makes it useful for evading detection. Detecting malicious PowerShell scripts and commands is an open challenge for proactive endpoint protection due to three major issues: (1) The malicious commands are usually hidden in a long script beyond the processing limit of typical machine learning models. (2) They are usually mixed with bulky benign scripts. (3) Script obfuscation can easily conceal their potential matching signatures. In this article, we introduce a novel model addressing these challenges. It incorporates similarity learning, sentence transformer, sliding window method, and stochastic gradient descent (SGD) classifier. Our key insight is that malicious PowerShell code, particularly when obfuscated, exhibits semantic and statistical deviations from benign administrative usage, and these deviations can be captured by contrastive sentence embeddings without the need for de-obfuscation or handcrafted features. We operate this insight through a Siamese similarity learning framework that improves robustness against Out-of-Vocabulary tokens due to unseen code obfuscation methods. The sliding window method enables the model to handle long scripts, and the SGD classifier evaluates segment-level maliciousness. Our model achieves accuracies of 99.01%, 97.59%, 98.70%, and 99.73% across multiple obfuscated and mixed script benchmarks, outperforming existing baselines by over 30% in all cases. This work demonstrates a scalable and effective strategy for robust PowerShell malware detection in real-world scenarios.