Generating Black-Box Adversarial Malware Examples Against Malware Detectors with Proximal Policy Optimization

Machine learning algorithms are becoming increasingly common in malware detection. However, these algorithms typically focus only on optimizing model accuracy and recall rate, without fully considering the fact that attackers may intentionally make changes to the malware so that generated adversarial malware can evade malware detectors while preserve the malicious functionality. This paper proposed a black-box adversarial malware generation method with Reinforcement Learning based on proximal policy optimization. The method takes actions based on learned policy to make changes to the origin malware. Compared to previous methods, our approach employed the PPO algorithm to enhance the stability of the model training process, and a more reasonable action space is designed to modify the origin malware. The experiment results show that it has over 94.8%-97.4% evasion rate for three state-of-the-art machine learning based detectors in a pure black-box setting.