Multi-objective Reinforcement Learning for PV-EV Integrated Energy Management

Due to the high volatility of photovoltaic energy and the flexible consumption characteristics of electric vehicles (EVs), the integration of EVs and photovoltaic systems into the existing grid presents both opportunities and challenges. This paper proposes a new two-layer integrated energy management framework to solve the difficulty of priority consumption of photovoltaic energy and improve the economic benefits of the power system. The framework optimizes power procurement and EV charging scheduling through a two-layer reinforcement learning approach. The first layer uses a deep Q-network (DQN) to make power purchasing decisions, giving priority to self-produced photovoltaic power and obtaining additional grid power only when necessary; The second layer uses a multi-objective reinforcement learning (MORL) approach with multiple DQN to manage service pricing and charging plans for various EV users. In addition, the weighted aggregation approach combines multiple Q values into a single policy to balance different user needs. Experimental results show that the framework significantly reduces operating costs, improves the efficiency of energy procurement and EV charging programs, and provides substantial economic incentives for renewable energy adoption and more efficient energy management.