A Digital-Twin-Aided Safe Multi-Agent Reinforcement Learning Framework for Renewable-Integrated Residential Energy Management

The increasing penetration of distributed renewable energy sources and electric vehicles (EVs) introduces significant operational challenges for residential energy management systems (HEMS), including stochastic renewable generation, uncertain load demand, device coupling, and physical safety constraints. This paper proposes a digital-twin-aided safe multi-agent reinforcement learning framework for coordinated energy management in renewable-integrated residential systems. The proposed approach models the battery energy storage system and the EV as independent agents and employs a multi-agent soft actor–critic (MASAC) algorithm with a centralised critic to capture the interactions among distributed energy resources. To improve decision quality under uncertainty, a digital twin module is developed to maintain a virtual representation of the residential energy system, synchronise operational states, update degradation-sensitive parameters, and generate short-term predictive information on photovoltaic (PV) generation and household load. The updated digital twin states and forecasts are incorporated into the observations of the reinforcement learning agents. In addition, a safety projection layer is incorporated to improve operational feasibility during both training and deployment. The environment considers realistic residential characteristics, including time-of-use electricity prices, battery degradation, EV mobility patterns, and grid energy trading. Simulation results show that the proposed framework reduces daily energy costs compared with rule-based baselines while maintaining EV charging reliability and operational feasibility. These results highlight the potential of combining predictive information, safety-constrained action execution, and multi-agent reinforcement learning for intelligent residential energy management.