Rapid Learning of Natural Gaits for Quadrupedal Locomotion and Skill Reuse in Downstream Tasks

Motion capture of quadrupedal animals can provide expert demonstration for the locomotion control of quadrupedal robots, whereas pure reinforcement learning using rewards based on human intuition often yield unnatural behaviors. This paper proposes a two-phase training framework to take advantage of such expert data for rapid style learning and flexible skill reuse. Generative adversarial imitation learning of gait styles and skill exploration via maximization of mutual information between skills and state transitions are conducted in parallel with reinforcement learning in the first pre-training phase. In the second phase, fine-tuned policy network for a customized downstream task is trained to realize skill reuse, even for a task not demonstrated in the expert data. The experiments on a physical robot have shown successful simulation-to-reality transfer, and skills learned from walking, trotting and turning of real animals on flat ground can yield robust behaviors like crawling on flat ground and climbing up stairs of the robot.