Adaptive Backstepping Optimal Tracking Control of Interconnected Robotic Manipulator System Based on Reinforcement Learning

In this paper, an optimal control method based on reinforcement learning (RL) is proposed for the tracking control problem of robotic manipulator (RM) system. In contemporary industrial manufacturing and precision technology, RM has gained significant popularity. To solve the optimal tracking control problem for RM, its dynamics are decomposed into interconnected subsystems. Combining the backstepping method and the RL method, both the virtual control signal and the actual control signal are designed as the optimal solution of each error subsystem, while ensuring that the virtual controller and the actual controller of all subsystems are optimal. For the inherent nonlinearity and difficult solution of the Hamilton–Jacobi–Bellman (HJB) equation, an actor-critic neural network (NN) is constructed to approximate the nonlinear term, and a cost function with input gain function is introduced to ensure the tracking effect. Then, based on the Lyapunov stability theory, it is proved that all the error signals of the system are semi-globally uniformly ultimately bounded (SGUUB). Finally, the effectiveness of the proposed method is validated through 2-degree-of-freedom (DOF) and 3-DOF RM simulations, with robustness verification against sudden disturbances. Note to Practitioners—The tracking control problem of RM is a very extensive and important problem in industrial applications, such as delivery, grasping, etc. The optimal control theory can solve the problems of shortest path, least fuel and shortest time. When RM perform different tasks, different cost functions can be constructed according to the control objectives. Most of the existing optimal control methods for RM systems cannot guarantee the optimality of the entire controller. Inspired by this, this paper develops a new RM optimal tracking control strategy, which can ensure that the virtual controller and the actual controller are the optimal solution, so as to ensure the overall optimality. Based on the RL method, the problem of solving the HJB equation is solved, in which the actor-critic NN weights are updated online at the same time. The actor NN is used to execute control actions, and the critic NN evaluates the control actions and feeds back to the actor NN to improve the subsequent control input.