Hierarchical smart charging algorithm for EVs based on multi-agent reinforcement learning

With the large-scale integration of electric vehicles, disorderly charging behaviors pose a serious challenge to the stability of the power grid. This paper proposes an electric vehicle hierarchical smart charging algorithm based on multi-agent reinforcement learning (HSCA-MARL) to resolve the conflict between individual interests and system constraints in the V2G scenario with only one power transformer. The algorithm adopts a hierarchical hybrid architecture, including the distributed agents, a global coordination agent, and a contribution-driven value transfer mechanism. Each charging station acts as an independent agent and the individual object is optimized through the local Actor-Critic network. The global Critic network quantifies system risks, and the contribution-driven mechanism distributes global Q value based on the contribution of user satisfaction, economic benefits, and load balance. Each individual's learning is guided by combining global Q value and local Q value, so as to balance team goals and individual goals. Experimental results show that the proposed algorithm outperforms other algorithms in terms of algorithm convergence, global return, and variance of individual return.