IM-TD3: A Reinforcement Learning Approach for Liquid Rocket Engine Start-Up Optimization

With advancements in reusable liquid rocket engine technology to meet the diverse demands of space missions, engine systems have become increasingly complex. In most cases, these engines rely on stable open-loop control and closed-loop regulation systems. However, due to the high degree of coupling and nonlinear dynamics within the system, most transient adjustments still depend on open-loop control. Open-loop control often fails to provide the optimal control strategy when encountering external disturbances. To address this issue, we introduce the intrinsically motivated twin delayed deep deterministic (TD3) algorithm, specifically designed for the startup process of LOX/Kerosene high-pressure staged combustion engine. This approach leverages intrinsic motivation to enable the algorithm to adapt to the abrupt parameter changes during the start-up process. A series of comprehensive experiments were conducted to verify the effectiveness of our method. The experimental results demonstrate that our method outperforms both the PID method and previous researchers' reinforcement learning methods based on the TD3 algorithm and DDPG, achieving a faster and more stable start-up process and significantly enhancing engine performance.