Multi-step Prioritized Deep Reinforcement Learning Method for Home Energy Management

Driven by the progress in energy scheduling technologies and the ongoing refinement of the electricity market, an increasing number of residential users are actively participating in demand response (DR) through home energy management systems (HEMS) to minimize energy costs. However, traditional model-based optimization methods face challenges in modeling accuracy due to the growing diversity of smart devices in households. Additionally, the uncertainties of photovoltaic generation and electricity prices reduce the efficiency and reliability of scheduling strategies. To address these challenges, this paper first establishes a home energy system composed of photovoltaic system, energy storage, and household appliances, which is then formulated into a Markov Decision Process (MDP). Subsequently, a multi-step twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm with a prioritized experience replay (PER) is proposed, which aims to minimize electricity costs for residential users by coordinating and optimizing energy generation, consumption, and storage. This approach effectively addresses the issues of modeling accuracy and uncertainty in the home energy system. Finally, simulation experiments are conducted and compared with baseline algorithms, demonstrating that our proposed method can better coordinate and optimize various household smart devices while reducing electricity costs.