Generative Adversarial Imitation Learning Method Based on TD3-SAC Hybrid Algorithm for Robot Motion Control

The dependency on expert demonstrations and the difficulty in surpassing expert performance have been significant limitations in the field of imitation learning. This paper presents a TD3-SAC Hybrid Method that combines the advantages of two advanced RL algorithms, TD3 and SAC, integrating their exploration mechanisms to enhance the exploration capability of Generative Adversarial Imitation Learning (GAIL, an imitation learning method using generative adversarial networks). Furthermore, we designed a closed-loop method for generating expert demonstrations through a learning-based data generation and feedback approach, and proposed a GAIL model training pipeline based on TD3-SAC Hybrid Method, ultimately forming a robust robot motion control method. Through comparative experiments, we demonstrated the effectiveness of the TD3- SAC hybrid method in GAIL. It outperforms baselines such as C-GAIL, GDSG, DAC, and methods combining classical RL algorithms with GAIL, achieving better performance, overcoming the limitations of expert demonstrations and demonstrating a higher probability of outperforming expert demonstrations in MuJoCo simulation environments (46% in HalfCheetah, 35% in LunarLander, 14% in Walker2d, and 21% in Hopper). This method not only retains the advantages of imitation learning but also equips the agent with the necessary exploration mechanisms to overcome the limitations of expert-based learning, particularly in challenging and complex environments.