Hierarchical reinforcement learning for enhancing vehicle-to-grid operations in solar power systems

This study proposes a Hierarchical Reinforcement Learning (HRL)-based framework for optimizing Vehicle-to-Grid (V2G) operations, with a focus on addressing solar power generation (SPG) forecast errors and facilitating efficient energy exchange between electric vehicles (EVs) and the grid. To tackle the inherent variability of solar energy and dynamic operational scenarios, the proposed approach formulates V2G operations as a multi-level decision-making process. High-level policies are designed to handle global objectives such as grid stability, while low-level policies focus on optimizing battery-level operations. Leveraging simulated SPG data and synthetic forecasts, the HRL framework provides a robust environment for training and evaluation, enabling a detailed exploration of critical parameters like blocking probability thresholds and discount factors. Results show significant reductions in Mean Squared Error (MSE) values and improved SPG forecast accuracy, highlighting the model’s adaptability to dynamic conditions. Training loss trends confirm convergence, and sensitivity analyses underscore the framework’s flexibility under diverse operational scenarios. The findings demonstrate the potential of HRL-based strategies to enhance smart grid efficiency by seamlessly integrating distributed energy resources (DERs), offering a scalable and sustainable solution for modern energy management systems.