Data-Driven Reinforcement Learning for Energy Management with Peer-to-Peer Trading and Demand-Side Management

This paper presents demand side management (DSM) for integrated energy systems of dairy farms - in order to lower the energy consumption - utilizing a variety of energy devices and load types, in addition to Peer-to-peer (P2P) energy trading. The decision tree regression model is used to anticipate the day-ahead PV power generation, utility grid energy pricing, and various loads consumption based on real-world data in order to implement DSM and P2P energy trading (P2P-ET). To handle the growing uncertainties - brought on by bidding actions, transaction volumes, and forecasted data on load profiles, the generation of renewable resources, and energy prices - a modified multi-agent reinforcement learning (MARL) is used for decision-making. To address such a situation, the DSM and P2P-ET problem is formulated as a finite Markov decision process. The mixed uncertainty is included as additional stochastically represented states and action scenarios in the modified MARL. It is found that incorporating additional stochastic states and action scenarios significantly enhances the RL agent's ability to adapt to diverse and dynamic environments, particularly when faced with unexpected variations in PV generation and energy price. These extra states and actions allow the agent to learn more nuanced strategies and respond effectively to probabilistic circumstances. The simulation result unveil that by using the proposed MARL algorithm to optimize the P2P-ET and DSM strategies, the average load can be reduced by 20.73%. Moreover,, the optimal P2P-ET results in buying 82.1% of the energy needed from the P2P community, while the remaining 17.9% is bought from the utility grid. Finally, by applying both the optimal DSM and P2P-ET, the average daily cost of energy can be reduced by 23.57%