Battery Health-Informed and Policy-Aware Deep Reinforcement Learning for EV-Facilitated Distribution Grid Optimal Policy

Electric vehicles play a crucial role in modern active distribution system operation, owing to their flexibility and mobility. The joint optimization of power grid operation and EV scheduling is expected to achieve better decision-making for both power and transportation systems. However, such joint optimization can suffer from high computational complexity and a lack of information about the other system. One possible approach for the power system operator is to assume the charging/discharging pattern of the EV population and make dispatch decisions based on such estimated EV profiles. This practice results in grid-level policies. The problem arises as whether the EV population complies with and fulfills the grid-level policies is not easy to monitor. To address this issue, we further investigate and expand upon the innovative concept of grid facilitation for electric vehicles (EVs). A newly designed probabilistic incentive signal is introduced to assist in implementing grid policies. Correspondingly, we develop a deep reinforcement learning (DRL) framework for EV prosumer scheduling that effectively balances EV autonomy with the execution of grid policies. Cast as a demand response signal, the grid-level policy is capable of managing overall grid operation with the facilitation of individuals and decoupling the co-optimization of the operator and EV owner. For the DRL of the EV, the training reward considers both the EV operation cost and the grid facilitation incentive, featuring a dynamically shrinking EV battery’s feasible operation range in terms of voltages and currents, and the health-informed reward. The policy-aware reward utilizes the Jensen-Shannon divergence to quantify the gap between the actual EV power injection pattern and the system operator’s assumption. Invalid action mask tricks are integrated to prevent infeasible actions for EVs. The simulation results demonstrate the effectiveness of the proposed framework and the facilitation effect of EV on system operation planning.