Rule-Guided DRL for UAV-Assisted Wireless Sensor Networks With No-Fly Zones Safety

Essential safety constraints are critical to data collection in UAV-assisted wireless sensor networks (WSNs). This paper proposes a dynamic model for such WSNs considering safety of no-fly zones (NFZs) and uncertainty of sensor data. Under the dual objectives of minimizing the age of information (AoI) and reducing packet loss, we formulate the UAV trajectory optimization problem with NFZ constraints as a constraint Markov decision process (CMDP). To integrate safety into the exploring and learning process, we propose a rule-guided deep reinforcement learning (RG-DRL) scheme, providing safety guarantee and robust performance for the off-policy DRL agent. On one hand, a rule-based AI method is employed to strategically guide the convergence of the DRL agent. On the other hand, a novel safety technique named differentiable algorithm enabled safety layer (DASLayer) is introduced to navigate the UAV in complex environments without violating safety constraints. By relaxing the artificial potential field (APF) algorithm into a differentiable manner, the DASLayer is compatible with common neural architectures, facilitating simultaneous training alongside the neural network. Statistical results demonstrate the effectiveness of the proposed method, ensuring zero safety constraint violations, and yielding significant improvements in AoI (25%) and packet loss reduction (30%).