Safe reinforcement learning for ship energy management optimization with LLM-based reward shaping

Cruise ships are large greenhouse gas emitters. Given the net-zero emission goal by 2050 set by the International Maritime Organization, it is crucial to focus on different strategies for reducing emissions. One promising strategy is to work on reducing energy losses due to non-optimal management of a cruise ship’s complex energy system. Effectively managing a ship’s energy consumption is a difficult task. Modern reinforcement learning approaches have been employed to the task at hand. Results have shown improvements over traditional optimization methods that fail to scale to large stochastic problems such as ship energy optimization. However, reinforcement learning approaches lack a direct focus on ensuring the safety of the system, which is critical for a real-world application.This paper showcases work in progress related to developing a safe reinforcement learning (RL) framework for ship energy optimization. We utilize a formal safety shield to block unsafe control actions and a large language model (LLM) to generate a reward function (RF) that incorporates safety constraints. We construct a white-box physical system of a ship’s energy management system and a faster black-box surrogate model as environments to train and test our RL methods. Preliminary results show that the integration of the safety shield and the LLM-generated reward function is promising in reducing safety violations.