Toward a Safe Accommodation to Faults in Control Loops via Reinforcement Learning and Reachability Under Bounded Variations

A wider deployment of learning-enabled control loops in aerospace systems requires to pay special attention to safety and fault tolerance, in particular when targeting a safe accommodation to faults and disturbances. In this paper, a training-independent offline safety verification of control loops hybridizing a baseline model-based controller and a constrained reinforcement learning agent is considered. To this purpose, a dedicated reachability algorithm based on zonotopes is proposed to deal with bounded inputs with bounded variations. This formulation contributes to reducing conservatism while still covering a significant range of physical phenomena that are not arbitrarily fast, like surface control runaway or Oscillatory Failure Cases (OFC). The learning agent is then trained to improve given performance metrics under non-modeled environmental dynamics, including faulty scenarios. A safe accommodation scheme is so obtained and exemplified through numerical simulations using the Civilian Aircraft Landing Challenge (CALC) benchmark.