Efficient microgrid scheduling via actor-critic reinforcement learning with minimal state representation and well-defined reward design

The energy management problem of a microgrid (MG) involves finding a sequence of decisions under uncertainty, primarily due to the randomness in generation from renewable energy sources, load demand and price profile. Among the Reinforcement Learning algorithms, the actor-critic algorithm stands out due to its ability to capture the advantages of both value-based and policy-based methods. In this paper, we propose an actor-critic algorithm for the energy management of a grid-connected microgrid with renewable sources, a microturbine, battery energy storage and load. The algorithm employs two readily available state variables and a well-suited reward function to reduce the complexity and dimensionality issues. The algorithm’s optimisation effectiveness under varying renewable energy source generation, load demand and price profile with a maximum deviation of 10% has been evaluated, and robustness is checked for a sudden change in tariff with an unknown distribution after the learning phase. A detailed comparison with existing works in terms of state variables, dimensionality, convergence efficiency and computational time is presented and discussed. With the reduced state variable approach proposed in this work, the algorithm converges to the optimal solution with a computational time which is comparatively lowest among the other methods. The cost of operation of the microgrid undergoes a 69% reduction with this method.