Autonomous navigation in realistic urban environments using machine learning and path planning

Deep Reinforcement Learning (DRL) has achieved remarkable results in simulated environments for its ability to learn optimal actions through interaction. Yet, applying these techniques to complex urban contexts remains challenging due to the need for efficient exploration and adaptive decision-making. This work applies DRL to autonomous navigation in a realistic simulated environment based on Vila Real, Portugal, comparing the Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithms. Alongside their implementation, the mathematical principles underlying both methods are examined to clarify their learning dynamics. The PPO agent achieved a best cumulative reward of 69.37 after 150 million training steps (66.1 hours), while SAC obtained -7.91 after 182.7 hours, showing limited performance. These results demonstrate the effectiveness of PPO-based DRL for autonomous navigation in complex urban environments.