Computational design exploration of rocket nozzle using deep reinforcement learning

• Study explores rocket nozzle design using deep reinforcement learning (DRL) combined with CFD simulations to generate novel geometries. • DRL agents with varying degrees of freedom (DOF) optimize nozzle shapes, while a CNN-LSTM model accelerates design iterations by predicting Mach numbers from images. • DS2 nozzles achieved better efficiency (lower surface area and volume) than minimum length nozzles, showcasing DRL's potential for innovative designs. • Replacing CFD with machine learning models and extending to 3D nozzle designs for faster and broader exploration. Deep Reinforcement Learning (DRL) has emerged as a powerful tool for solving high-dimensional optimization problems in complex, unexplored domains. This study presents a novel application of DRL for computational design exploration of rocket nozzles, demonstrating its superiority over traditional parametric optimization techniques. The DRL-generated nozzles achieve a 10.5 % reduction in surface area and an 11.8 % reduction in volume compared to analytically designed minimum-length nozzles (MLNs), while maintaining similar Mach numbers at the outlet. This underscores DRL's ability to autonomously explore unconventional geometries, leading to significant improvements in structural efficiency and material savings. A key novelty of this work is the integration of a CNN-LSTM-based surrogate model to accelerate the design process by predicting Mach numbers from nozzle images, significantly reducing reliance on computationally expensive CFD simulations. This hybrid approach not only expedites the optimization cycle but also enables real-time design iterations. Additionally, the use of the Single-Step Proximal Policy Optimization (SSPPO) algorithm enhances the exploration of nozzle geometries by maximizing aerodynamic performance while balancing computational efficiency. This research establishes DRL as a promising optimization framework for aerospace applications, particularly in domains where traditional methods struggle with geometric complexity. The ability of DRL agents to autonomously generate and refine nozzle geometries, coupled with deep learning-based predictive modeling, paves the way for rapid, cost-effective, and automated aerospace component development. These findings highlight the transformative potential of DRL in computational engineering, setting a foundation for future advancements in propulsion system design.