Optimizing Oil Field Development with Multi-Action Decision-Making in Deep Reinforcement Learning

Abstract Field Development Planning (FDP) has traditionally relied on conventional optimization techniques that are often static and struggle to adapt to the dynamic and uncertain nature of reservoir management. To address these shortcomings, this study develops a multi-action deep reinforcement learning (DRL) framework for autonomous and adaptive FDP. The approach utilizes the Proximal Policy Optimization (PPO) algorithm to train an agent that makes simultaneous decisions on well placement and well type (producer or injector) within a two-dimensional, two-phase reservoir environment integrated with a commercial simulator. The agent's objective is to learn a policy that maximizes the Net Present Value (NPV) over a four-year development period with yearly decisions. The results demonstrate that the agent successfully learns to optimize its strategy, with the mean reward increasing from approximately $12.5 million to over $20 million during training. On 100 unseen test cases, the trained agent achieved a mean NPV of $19.3 million, a value nearly three times higher than the best-performing conventional benchmark, the 4-spot pattern. The agent autonomously learned a robust strategy, favoring a 3:1 producer-to-injector ratio, on average, and discovered geologically sound well-spacing principles to optimize drainage and sweep efficiency. This work underscores the significant economic and strategic advantages of using a DRL-based approach, which can generate data-driven development plans that are superior to traditional, predefined strategies.