Efficient Deep Reinforcement Learning-Based Coordination for Unlocking Electric Vehicle Decarbonization Potential in Coupled Transportation and Power Networks

As a part of the global decarbonization agenda, the electrification of the transport sector involving the large-scale integration of electric vehicles (EV) constitutes one of the key initiatives. However, the decarbonization potential of EV cannot be exploited without appropriate incentive and coordination. Deep reinforcement learning (DRL) constitutes a well-suited model-free and data-driven framework to coordinate EV's charging decisions. Its real-world application facing multiple uncertainties is still challenging, due to the limited interaction efficiency between agent and environment of existing approaches. Therefore, this paper proposes a novel DRL-based coordination method, employing a pre-trained edge conditioned convolutional network and deep belief network as surrogate training environment to speed up the interaction, and combining a learning acceleration mechanism which enhances the exploration capabilities. This method is complemented in coupled transportation and power network (CTPN). Agent learns the optimal charging price composed of energy price and carbon obligation price, and incentivizes EV low-carbon coordination. Case studies involving a real-world scale CTPN are designed and the results demonstrate the effectiveness of the proposed coordination method in mitigating the operational cost and global carbon emission. The proposed method is also proved to outperform the state-of-the-art DRL methods in terms of the computational efficiency and generalization ability.