AV evasion techniques: A practical evaluation of payload obfuscation using MSFvenom, Veil, Empire and FATRAT

Antivirus (AV) programs play an essential role in defending today’s digital systems, acting as a first line of protection against malware by detecting, blocking, and neutralizing threats. Yet, as cyber defenses have evolved, so too have the tactics used by adversaries. Skilled attackers and ethical hacking teams are increasingly turning to evasion strategies designed to slip past antivirus mechanisms. Tools such as msfvenom and the Veil Framework are commonly used to craft payloads that are disguised well enough to evade both static and behavioral detection methods. This study explores the practical effectiveness of these AV evasion techniques within a controlled lab environment, using Windows Defender as the focus of the analysis. The core aim is to observe and evaluate how conventional, signature-based AV engines respond when exposed to both unmodified and obfuscated payloads—especially those crafted using msfvenom and later enhanced with Veil. All testing is conducted within a sandboxed setup to ensure safety and ethical boundaries are respected. The research outlines the methodology in detail, including the generation of reverse shell payloads, multi-layered obfuscation, and analysis of antivirus reactions to different variants. By capturing and analyzing this behavior, the study aims to provide deeper insight into the current limitations of endpoint security solutions and highlight techniques attackers use to bypass them. The results are expected to offer practical value for cybersecurity professionals, red teamers, and students focused on malware analysis and adversarial simulation.