Integrated DDPG-PSO energy management systems for enhanced battery cycling and efficient grid utilization

• Integrating the DDPG's RL capabilities in exploitation with PSO's global search efficiency to improve optimization solutions quality via superior exploration and exploitation stages. • Leverage DDPG's RL capabilities to achieve a robust EMS algorithm for energy environment with stochastic renewable resources and increased system complexity in mid and long-term energy optimization problems. • Utilizing the PSO to optimize actor network parameters in the DDPG framework, resulting in optimal shallow battery scheduling compared to DDPG's deep cycling for extending the battery's lifespan. • Introducing flexibility in specifying actions has enhanced the effectiveness of the DDPG-PSO EMS, enabling it to significantly reduce grid capacity requirements and utilization compared to standalone DDPG and other metaheuristic methods. Effective energy management is crucial in hybrid energy systems for optimal resource utilization and cost savings. This study integrates Deep Deterministic Policy Gradient (DDPG) with Particle Swarm Optimization (PSO) to enhance exploration and exploitation in the optimization process, aiming to improve energy resource utilization and reduce costs in hybrid energy systems. The integrated DDPG-PSO approach leverages DDPG's reinforcement learning and PSO's global search capabilities to enhance optimization solution quality. The PSO optimizes the DDPG actor-network parameters, providing a strong initial policy. DDPG then fine-tunes these parameters by interacting with the energy system, making decisions on battery scheduling and grid usage to maximize cost rewards. The results show that the integrated DDPG-PSO EMS outperforms the traditional DDPG in terms of battery scheduling and grid utilization efficiency. Cost evaluations under critical peak tariffs indicate that both EMS algorithms achieved a 34 % cost saving compared to a grid-only system. Under differential grid tariffs, the proposed DDPG-PSO approach achieved a 28 % cost reduction, outperforming the standalone DDPG, which achieved a 25 % saving. Notably, the DDPG-PSO effectively reduced overall grid dependency, yielding a total operational cost of $665.19, compared to $780.70 for the DDPG. resenting a 14.8 % reduction. The battery charge/discharge profiles further highlight the advantages of the DDPG-PSO strategy. It demonstrated more stable and efficient energy flow behavior, characterized by shallow cycling and partial discharges sustained over several hours. In contrast, the DDPG exhibited more aggressive deep cycling, fluctuating frequently between minimum and maximum charge levels. This improved energy flow management by DDPG-PSO not only reduces wear on the battery system but also promotes long-term sustainability and reliability in hybrid energy management.