Multi-Objective Reinforcement Learning with Shielding for Control and Safety in Power Grids

Traditional model-based methods relying on precise modeling are sensitive to errors and rapidly changing conditions, causing unstable control. Furthermore, real-world problems often involve conflicting objectives, requiring multi-objective reinforcement learning (MORL) in order to facilitate adaptability. However, MORL and single-objective RL (SORL) research often address these challenges separately due to the lack of a unified framework. To bridge this gap, we present eGridLVGym, an open-source collection of two test environments for managing low-voltage grids. The first environment models a household with rooftop photovoltaics, battery storage, electric vehicle charging, and a heat pump. The second extends this to a feeder grid with seven points of common coupling, representing diverse portfolios of distributed energy resources. These environments support system-wide flexibility dispatch while ensuring safety through soft and hard constraints that penalize or restrict unsafe actions. We evaluate eight state-of-the-art SORL algorithms (A2C, DDPG, PPO, SAC, TD3, Recurrent PPO, TQC, and TRPO), a grid code shielding approach, and the MORL algorithm CAPQL. Our results demonstrate that the best SORL algorithm reduces transformer overload from 93% to 75% and cuts economic costs to 122%, while the default-weighted MORL outperforms the rule-based controller by 77% in economic cost.