Water-Alternating-Gas and CO2 Storage Optimization Using Time-Lapse Geophysical Monitoring and Deep Reinforcement Learning

Abstract This study presents a new approach for optimizing water-alternating-gas (WAG) injection strategies and CO2 storage using deep reinforcement learning (DRL), supported by time-lapse gravity monitoring. We tested two reinforcement learning (RL) agents, Q-Learning (QL) and Double Deep Q-Network (DDQN), which interact with a high-fidelity reservoir simulation environment. The QL agent, despite its simplicity, demonstrates the fundamental concepts of optimal control of a WAG process using RL combined with gravity measurements. In contrast, the DDQN agent, combined with a convolutional neural network (CNN), outperforms other control methods by learning the spatial and temporal patterns of fluid movement within the subsurface. Comparisons with traditional, industry-standard WAG schedules reveal significant improvements in both Net Present Value (NPV) and CO2 storage efficiency using RL-optimized injection strategies. Time-lapse gravity data, simulated over 25 years, effectively capture and differentiate fluid displacement and accumulation under various injection regimes, making our proposed geophysical control approach possible. The industrial WAG schedule, involving regular switching between water and gas phases, results in suboptimal CO2 trapping due to early gas breakthrough and limited dissolution. Conversely, the DDQN-optimized policy enables longer CO2 retention in the reservoir, promoting greater dissolution into the formation brine and improving long-term geological storage. These findings underscore the importance of intelligent control strategies and advanced state representations in optimizing both economic and environmental objectives for CO2-enhanced oil recovery and geological carbon storage operations.