Deep Reinforcement Learning-Based Energy Management Strategy for Green Ships Considering Photovoltaic Uncertainty

Owing to the global concern regarding fossil energy consumption and carbon emissions, the power supply for traditional diesel-driven ships is being replaced by low-carbon power sources, which include hydrogen energy generation and photovoltaic (PV) power generation. However, the uncertainty of shipboard PV power generation due to weather changes and ship motion variations has become an essential factor restricting the energy management of all-electric ships. In this paper, a deep reinforcement learning-based optimization algorithm is proposed for a green ship energy management system (EMS) coupled with hydrogen fuel cells (HFCs), lithium batteries, PV generation, an electric power propulsion system, and service loads. The focus of this study is reducing the total operation cost and improving energy efficiency by jointly optimizing power generation and voyage scheduling, considering shipboard PV uncertainty. To verify the effectiveness of the proposed method, real-world data for a hybrid hydrogen- and PV-driven ship are selected for conducting case studies under various sailing conditions. The numerical results demonstrate that, compared to those obtained with the Double DQN algorithm, the PPO algorithm, and the DDPG algorithm without considering the PV system, the proposed DDPG algorithm reduces the total economic cost by 1.36%, 0.96%, and 4.42%, while effectively allocating power between the hydrogen fuel cell and the lithium battery and considering the uncertainty of on-board PV generation. The proposed approach can reduce energy waste and enhance economic benefits, sustainability, and green energy utilization while satisfying the energy demand for all-electric ships.