Reinforcement Learning-Based Time of Use Pricing Design Toward Distributed Energy Integration in Low Carbon Power System

Amidst the rapid transformation of the electricity supply and demand structure, there has been a consensus among policy makers on the need for further refinement of the time-of-use (ToU) pricing mechanism. Nonetheless, the challenge of capturing the dynamic interplay between ToU pricing design and the market behavior of diverse consumers, particularly in light of distributed energy integration, persists as an unresolved inquiry. This paper introduces a novel approach, utilizing reinforcement learning for the development of a ToU pricing model considering investment of distributed energy. The interaction between end consumers and utilities within the ToU pricing framework is encapsulated within a bi-level structure, characterized by significant applicability and scalability. A deep reinforcement learning algorithm is employed to train an agent in devising effective pricing strategies. To aid the agent in grasping the complex, interdependent pricing effects during peak, mid-peak, and valley periods, a configuration employing three interconnected Long Short-Term Memory networks is adopted. Case studies, grounded in empirical datasets, substantiate the efficacy and rationality of the methodology presented herein. It is anticipated that the framework proposed in this study will serve as a valuable reference for the design of efficient ToU pricing in diverse regions.