Boosting Deep Reinforcement Learning-Based Path Planning for Robotic Manipulators With Egocentric State Space Descriptions

In robotic path planning tasks, Reinforcement Learning agents typically receive global or relative Euclidean coordinates, e.g., with respect to a target reference point as direct state information. Nevertheless, a more egocentric view of the environment seems to be favorable – based on information in polar or spherical coordinates about objects surrounding the robot. Using the model-free, actor-critic algorithm Twin Delayed Deep Deterministic Policy Gradient in combination with Prioritized Experience Replay, the advantages of an alternative definition of states using egocentric TCP-coordinates is evaluated and compared in simulations to classical approaches within two typical environments. The training results indicate a tremendous potential of the egocentric state space definition that not only offers faster learning but also more successful trainings.