Enhancing autonomous navigation through hybrid DQN optimization: integrating prioritized replay and simulated annealing in dynamic environments

Abstract Autonomous navigation in unstructured environments remains a critical challenge in mobile robotics, where conventional planners such as A* and potential field methods exhibit limited adaptability in dynamic scenarios. This study presents an enhanced deep Q-network (DQN) architecture incorporating three synergistic innovations: (1) a prioritized experience replay mechanism that uses time difference-error weighted sampling to accelerate policy convergence, achieving a 38.7% speedup compared to vanilla DQN; (2) a hybrid action space framework combining discrete directional choices (8 headings) with continuous velocity control (0–2 m s −1 resolution), enabling collision avoidance under proximity constraints as tight as 0.3 m; (3) a simulated annealing-inspired exploration strategy with adaptive temperature decay, which dynamically balances the exploration–exploitation tradeoff. Extensive experiments across six benchmark environments demonstrate the superiority of the proposed method: it achieves 21.3% shorter path lengths than RRT*, a 63.4% higher success rate than traditional DQN in cluttered dynamic settings, and notably, a 92.7% improvement in computational efficiency over A* for real-time replanning tasks. These advancements lay a robust foundation for deploying autonomous robots in logistics and disaster response applications.