Global Path Planning for Amphibious Robots Based on Improved Q-Learning

Aiming at the problems of excessive energy consumption, slow convergence speed and long running time of amphibious robots in 3D environment, this paper presents an optimized path planning method using improved Q-learning for amphibious robots. The algorithm combines the Dyna framework with Q-learning, which integrates modelling, learning and planning to boost the algorithm's rate of convergence. Meanwhile, the artificial potential field method is introduced to initialize the Q-value by using the potential value of the environment as the search inspiration information to improve the convergence speed. The dynamic exploration factor is introduced to make the algorithm quickly adapt to the environmental changes and improve optimization efficiency. Finally, considering the different energy consumption of movement in different environments, the reward function is crafted to facilitate the amphibious robot to choose an optimal path with the lowest energy consumption. According to the simulation results, the optimized algorithm accelerates the convergence speed, improves the learning efficiency, reduces the energy consumption, and enables the amphibious robot to quickly find a non-colliding trajectory with low energy consumption.