Low-Noise Path Planning for Urban Drone Missions: Acoustic Ray Tracing and DDPG Algorithm

Abstract This paper presents a comprehensive approach for mitigating noise pollution from unmanned aerial vehicles (UAVs) in urban environment through path planning using reinforcement learning (RL). The study focuses on Turin, Italy, leveraging its diverse urban architecture to develop a comprehensive model. A detailed 3D occupancy grid map, based on OpenStreetMap data, was created to represent buildings’ locations and heights while a population density map was developed to account for demographic variances. The research develops a dynamic noise source model that adjusts noise emission levels based on UAV velocity, ensuring realistic noise impact predictions. Acoustic ray tracing techniques are utilised to simulate noise propagation, accounting for atmospheric absorption and reflections from urban structures, providing a detailed analysis of noise distribution. The core of this work is the application of the deep deterministic policy gradient (DDPG) algorithm within the RL framework. The algorithm is tailored to optimise flight paths by minimising noise impact while balancing other factors like path length and energy efficiency. The RL agent learns to navigate complex urban landscapes, integrating penalties for idling, excessive path length and abrupt manoeuvers to refine its path planning strategy. Simulation results with several maps unseen during training reveal that the RL-based approach effectively reduces noise impact in urban settings, making it a viable solution for better integrating UAVs into urban air mobility (UAM) systems. The methodology is scalable and adaptable, with potential applications in various urban environments globally. This research contributes to the development of sustainable drone operations in UAM context by addressing the critical issue of noise pollution, enhancing public acceptance and regulatory compliance.