Safety-Critical Learning of Robot Control With Temporal Logic Specifications

Reinforcement learning (RL) is a promising control approach in many scenarios. However, safety-critical applications are still a challenge due to lack of safety guarantees during exploration and subsequent deployment. The learning problem becomes even more difficult for complex tasks with temporal and logical constraints. In this article, we introduce a modular deep RL architecture as a control framework to satisfy complex tasks specified using linear temporal logic (LTL). To enhance safety, we propose a safe “shield” to constrain RL actions within a safe set. In turn, a challenge with this strategy is that the safe filter can restrict RL exploration during training. To address this limitation, we have developed an innovative safe guiding process by integrating the properties of LTL automata with perturbations of the safe “shield.” This innovation is verified to maintain the original optimality of LTL satisfaction and enhances exploration efficiency. Finally, we demonstrate that our approach consistently achieves near-perfect success rates and safety guarding with a high level of confidence during the training. The video demonstration can be found on our YouTube.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>