Safe Reinforcement Learning-Based Adaptive Hierarchical Control Approach for Virtual Coupling Trains

Virtual coupling presents a promising solution for enhancing urban rail transit efficiency through collaborative operations facilitated by information communication rather than physical connections. However, effectively controlling virtual coupling trains presents substantial challenges, particularly in managing implicit and nonlinear safety constraints. This paper proposes a novel adaptive hierarchical control approach to address these challenges, modeling the control problem as a Constrained Markov Decision Process (CMDP). The proposed approach integrates an upper controller utilizing safe reinforcement learning (RL) and a lower controller employing Proportional Integral Derivative (PID). Safe RL incorporates an unrolling safety layer (USL) into the RL algorithm, effectively addressing safety constraints inherent in the system. Additionally, dynamic adjustment of PID parameters based on the state of the virtual coupling trains enables precise control of train operations. This hierarchical architecture capitalizes on the adaptability of safe RL while leveraging the stability of PID, facilitating dynamic strategy adjustments in response to environmental changes to ensure virtual coupling stability and reliability. Simulation experiments validate the efficacy of the proposed approach, demonstrating significant advantages in controlling virtual coupling trains and thereby enhancing operational efficiency and safety.