Reinforcement Learning for Mitigating Malware Propagation in Wireless Radar Sensor Networks with Channel Modeling

With the rapid development of research on Wireless Radar Sensor Networks (WRSNs), security issues have become a major challenge. Recent studies have highlighted numerous security threats in WRSNs. Given their widespread application value, the operational security of WRSNs needs to be ensured. This study focuses on the problem of malware propagation in WRSNs. In this study, the complex characteristics of WRSNs are considered to construct the epidemic VCISQ model. The model incorporates necessary factors such as node density, Rayleigh fading channels, and time delay, which were often overlooked in previous studies. This model achieves a breakthrough in accurately describing real-world scenarios of malware propagation in WRSNs. To control malware spread, a hybrid control strategy combining quarantine and patching measures are introduced. In addition, the optimal control method is used to minimize control costs. Considering the robustness and adaptability of the control method, two model-free reinforcement learning (RL) strategies are proposed: Proximal Policy Optimization (PPO) and Multi-Agent Proximal Policy Optimization (MAPPO). These strategies reformulate the original optimal control problem as a Markov decision process. To demonstrate the superiority of our approach, multi-dimensional ablation studies and numerical experiments are conducted. The results show that the hybrid control strategy outperforms single strategies in suppressing malware propagation and reducing costs. Furthermore, the experiments reveal the significant impact of time delays on the dynamics of the VCISQ model and control effectiveness. Finally, the PPO and MAPPO algorithms demonstrate superior performance in control costs and convergence compared to traditional RL algorithms. This highlights their effectiveness in addressing malware propagation in WRSNs.