Adaptive undulatory locomotion of snake-like robots in dynamic viscous environments via deep reinforcement learning

This paper demonstrates how deep reinforcement learning (DRL) enables adaptive locomotion of snake-like robots in dynamically changing viscous environments. Traditional control methods exhibit inherent performance limitations in such environments. The fundamental challenge of this task is the necessity to adapt to fluid properties that are unobservable with local onboard sensors. To overcome this, we formulate the problem as a partially observable Markov decision process and solve it using an asymmetric actor-critic framework. In this approach, a teacher policy trained using privileged information available only in the physics simulator distills its knowledge into a student policy that relies on proprioceptive sensor information. Simulation results across a wide range of dynamic viscosity changes (10−7 to 10−2 m2/s) reveal that the DRL agent autonomously acquires non-sinusoidal adaptive gaits. These gaits improve propulsion velocity and transport efficiency, breaking the inherent limits of conventional sinusoidal and kinematic control. The findings establish that implicit environment inference via privileged information distillation is an effective approach to bypass the constraints of traditional models under unpredictable fluid dynamics.