Navigation and Anti-Interference Policies for Unmanned Vehicles in Noisy Unknown Environments Based on DDPG

Deep reinforcement learning shows great potential in the autonomous navigation of unmanned vehicles. This study focuses on analyzing the impact of environmental noise, specifically Gaussian, impulse, and uniform noise, on the waypoint navigation and obstacle avoidance performance of unmanned vehicles based on the Deep Deterministic Policy Gradient (DDPG) algorithm. We employed the Deep Deterministic Policy Gradient (DDPG) algorithm to train agents in environments with Gaussian noise characterized by unknown non-zero mean and standard deviation, and evaluated their performance under various noise types and conditions. In each simulation, the environment randomly generates target points to simulate the impact of noise in real sensor measurements. We find that denoising techniques such as low-pass filters and Kalman filters can effectively improve the performance of unmanned vehicles in the presence of unbiased Gaussian noise, while their effectiveness varies for other noise types. Notably, injecting noise during evaluation can sometimes enhance the performance of unmanned vehicles, potentially by acting as a form of regularization or helping the deterministic policy escape suboptimal, brittle trajectories. We conduct extensive training and testing of the deep reinforcement learning agent in the Gazebo physics simulator. To verify the effectiveness of the proposed method, we directly transfer the trained policy to a real unmanned vehicle without further modification and validate the results in a real environment. Our research provides new insights into understanding the impact of different noise types on the performance of unmanned vehicles based on deep reinforcement learning and offers a valuable reference for reliable navigation of unmanned vehicles in complex environments.