Intelligent coalbed methane drainage optimization: A deep reinforcement learning-driven life-cycle strategy

• Develops an efficient surrogate model: A CNN-LSTM-based surrogate model is constructed, accelerating simulations by orders of magnitude with high fidelity (average R² > 0.98), which replaces computationally expensive physics-based models and enables effective DRL training. • Pioneers the use of Deep Reinforcement Learning (DRL): The study innovatively applies a DRL framework to automate the life-cycle drainage optimization for Coalbed Methane (CBM) wells, allowing an intelligent agent to learn optimal strategies through interaction with a virtual environment. • Compares distinct optimization goals: Two DRL strategies are evaluated based on different rewards: maximizing gas production versus maximizing Net Present Value (NPV). The NPV-driven agent learns a more sophisticated policy that balances aggressive early drainage for revenue with conservative later drainage to minimize costs. • Demonstrates significant economic uplift: The NPV-driven DRL strategy increases the project's cumulative NPV by 5.83% compared to the historical strategy, proving the method's superior economic performance and its practical value for optimizing unconventional gas resource development. Coalbed methane (CBM) production, as a significant portion of unconventional natural gas development, plays a crucial role in enhancing output and economic benefits through the optimization of its life-cycle drainage strategy. Traditional drainage strategies rely on experience and trial-and-error methods, making it difficult to adapt to complex and dynamic production environments. This study proposes a life-cycle production and drainage optimization strategy for CBM based on Deep Reinforcement Learning (DRL). Utilizing the Deep Q-Network (DQN) algorithm, this work learns and optimizes the drainage strategy during the production process, achieving intelligent decision-making for drainage operations. An auto-regressive surrogate model is introduced to build a reinforcement learning environment for gas production and drainage, based on a deep learning model (CNN-LSTM). This model substitutes the full-physics simulation model that requires high computational cost, significantly accelerating the interactive learning process between the agent and the environment in DRL. Whether to set the gas production or Net Present Value (NPV) as reward, two strategies for reinforcement learning were considered accordingly. The results concluded that the DRL drainage strategy with NPV as the reward increased the net gain by 5.83% compared to historical data. Compared with traditional methods, this approach significantly improves the NPV and optimizes the drainage strategy. The findings demonstrate that the life-cycle drainage optimization method for CBM based on DRL is not only efficient and feasible but also provides an intelligent solution for the development of unconventional natural gas resources. The results highlight the method's strong adaptability and potential for addressing complex optimization problems in dynamic production environments.