Robust Deep Reinforcement Learning Control for a Launcher Upper Stage Module with Stability Certificate

This paper considers the design of an attitude controller of a launcher upper stage module during its exoatmospheric phase, where short boosts are performed to adapt the flight path. Those maneuvers can increase propellants’ motion in the tanks, leading to the so-called sloshing phenomenon that may affect the stability of the vehicle.A Deep Reinforcement Learning (DRL) algorithm is proposed to design the controller accounting for non linearities of the launcher and sloshing dynamics as well as presence of time-delay, bias and saturations on the actuation system. Based on Proximal Policy Optimization (PPO) and Almost Lyapunov functions in an actor-critic scheme, it allows to robustly learn a controller along with stability certificates, in presence of model uncertainties. Simulation results are proposed to illustrate the approach.