Cascade-trained deep reinforcement learning for PID gain optimization in precise joint position control of 6-DoF robotic arm

Abstract The problem of precise joint position tracking has remained as a core challenge for a 6-DoF Cobot arm, especially due to often scenario of an arbitrary waypoint reference trajectory generated due to human interactions. To tackle this problem, we propose a novel cascade training based Deep Reinforcement Learning (DRL) algorithm that tunes the PID controller gains for each joint simultaneously, ensuring accurate positional tracking for all joints. This also addresses the problem of overestimation of control parameters by ensuring that performance criteria are met in a phased manner during the training process. The tuned DRL based PID clearly outperforms the conventional PID control by accurately tracking the arbitrary waypoints given to each joints of the Cobot arm. We show the efficacy of the proposed method through exhaustive simulations and performing quantitative analysis of various key performance criteria like- Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Average Control Effort (ACE) error for the Cobot. The obtained results of DRL-PID control, when compared with its conventional PID counterpart, clearly depict the superiority of the proposed DRL-PID scheme via a cascade training approach. We have also remarked on some trade off and implementation aspects of the proposed control policy for the Cobot based applications. This method has the potential to be applicable to similar complex dynamical systems like a Cobot, where arbitrary reference and human interactions are prime concerns.