Dynamic Optimization of SAGD Injection–Production Parameters Using Reinforcement Learning for Enhanced Thermal Recovery and Subcool Control

Abstract The effectiveness of steam assisted gravity drainage (SAGD) development is highly dependent on the stable development of the steam chamber and effective control for subcool to prevent steam breakthrough. To address these challenges, this study proposes a hybrid optimization and control framework for dynamically optimizing key injection–production parameters. A heterogeneous SAGD numerical model was established to capture the coupled thermal and multiphase flow behavior under varying operational conditions. Within this model, a hybrid control framework is proposed: the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is employed to determine long-term global control actions, including the steam injection rate and electric submersible pump (ESP) frequency in production intervals. On top of these global decisions, a proportional–integral–derivative (PID) controller is integrated to provide short-term, real-time adjustments of ESP frequency based on subcool feedback, ensuring thermal stability, and preventing steam breakthrough by maintaining subcool close to target value. The optimization results over a ten-year production period demonstrate that the proposed TD3-PID method not only improves economic and production performance but also enhances operational stability. Compared to the baseline scenario, the framework increased net present value (NPV) by 5.75%, enhanced oil recovery by 2.46%, and reduced the cumulative steam–oil ratio (cSOR) by 14.67%, while maintaining subcool within the desired operating range. To adjust the steam profile along the well section more precisely, as an extended study, inflow control valves (ICVs) were installed along the horizontal section of the injector to further refine steam distribution. In this configuration, TD3 continues to optimize global injection–production parameters, while PID controllers fine-tune individual ICV openings in real time based on subcool feedback. This refined control effectively directed more steam to low-permeability regions, leading to a more uniform steam chamber and improved thermal conformance. As a result, compared with the TD3-PID scheme without ICVs, NPV and oil recovery were increasd by 1.16% and 1.77%, respectively, albeit with a 12.23% increased in cSOR due to more steam costs, highlighting the trade-off between thermal balance and overall steam utilization efficiency. This study demonstrates the effectiveness of integrating reinforcement learning and feedback control to achieve coordinated global optimization and local stability in SAGD operations. By combining a TD3-based agent for parameters optimization with PID controllers for real-time operational adjustments, the framework addresses the challenges brought about by reservoir heterogeneity and dynamic operating conditions. This scalable and adaptive solution provides new insights for field-scale intelligent thermal recovery management and autonomous SAGD operations in complex reservoirs.