An Actor-Critic Reinforcement Learning Scheme for Reactive 3D Optimal Motion Planning Based on Fluid Dynamics

This work proposes a novel and provably correct method for three-dimensional optimal motion planning in complex environments. Our approach models the 3D motion planning problem by solving streamlines of the potential fluid flow, filling a gap in traditional motion planning techniques by guaranteeing a closed-loop, smooth and natural-looking navigation solution. Special emphasis is given to an inherent challenge of artificial potential field (APF) methods, namely establishing proofs of safety and stability over the entire optimization process. A model-based actor-critic reinforcement learning algorithm is introduced to approximate the optimal solution to the Hamilton-Jacobi-Bellman equation and update the controller parameters in a deterministic manner. Through a series of ROS-Gazebo software-in-the-loop simulations the proposed methodology demonstrates robustness and outperforms widely used methods such as the RRT<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup>, highlighting its contribution to the field of 3D optimal motion planning.