Reinforcement Learning-Based Dynamics Control of Biped Reconfigurable Robots for Standing Balancing Tasks

This paper presents a standing balance control scheme for biped reconfigurable robots (BRRs) via neuro-dynamic programming (NDP) method. First, a global dynamic model of BRRs is established, in which the torso model is approximated by utilizing the virtual model control (VMC). Then, the cost function is developed including the system state, control law, and the coupling torque between the torso and legs, to construct the optimal standing balance control framework. On the basis of NDP algorithm, the near-optimal standing balance control strategy is developed by designing the critic-only neural network to solve the Hamilton-Jacobi-Bellman equation. Then, the tracking error of closed-loop BRRs is proved to be uniformly ultimately bounded. Finally, a series of experimental studies are carried out on the self-designed hyper-redundant BRRs experimental platform to show the effectiveness of the proposed control method.