High-precision geosteering via reinforcement learning and particle filters

Abstract Geosteering, a key component of drilling operations, traditionally involves manual interpretation of various data sources such as well-log data. This introduces subjective biases and inconsistent procedures. Academic attempts to solve geosteering decision optimization with greedy optimization and approximate dynamic programming (ADP) showed promise but lacked adaptivity to realistic diverse scenarios. Reinforcement learning (RL) offers a solution to these challenges, facilitating optimal decision-making through reward-based iterative learning. State estimation methods, e.g., particle filter (PF), provide a complementary strategy for geosteering decision-making based on online information. We introduce RL-Estimation, a method that integrates an RL-based geosteering framework with PF to address realistic geosteering scenarios. RL-Estimation deploys PF to process real-time well-log data to estimate the location of the well relative to the stratigraphic layers, which then informs the RL-based decision-making process. Our findings indicate that RL-Estimation achieves at least 20% higher reservoir contact compared to using RL or PF alone. Additionally, RL-Estimation performs within 2% of the theoretically optimal benchmark, which assumes access to the true state instead of relying on estimates from PF. These results demonstrate the synergy between RL and PF, highlighting the method’s effectiveness and near-optimal performance in realistic geosteering scenarios.