3D Scheduling for UAV in Pesticide Spraying Based on Deep Reinforcement Learning

Crop protection UAVs face scheduling challenges in irregular hilly terrain due to topography, battery limits, variable-mass energy consumption, and spray costs. The core problem addressed in this study is how to jointly optimize the three-dimensional visiting sequence of heterogeneous spraying grids and the multi-mode mid-mission resupply decisions for a UAV under spatially uneven pest distributions. Specifically, this study aims to minimize the total operational economic cost—integrating energy consumption, pilot labor, and equipment depreciation—while strictly satisfying full-coverage and onboard resource safety constraints. Spatial heterogeneity in pest severity makes traditional reciprocating spraying ineffective for cost-optimal coverage. We subdivide the operational area into two grid sizes: large grids representing low-pest-infested zones requiring conventional spraying, and small grids indicating high-pest-infested areas demanding targeted enhanced spraying. We transform continuous full-area spraying into a sequential decision-making problem over swath-matched 3D grid centers, solved using a semantically decoupled dual-branch Double Dueling Deep Recurrent Q-Network (D3QN) to develop a three-dimensional variable-rate spraying schedule that minimizes overall costs while satisfying multiple physical constraints. A reward function centered on undiscounted total cost is designed, complemented by four active resupply strategies enabling end-to-end learning of scheduling decisions. A spatiotemporal decoupled network architecture combined with a beam search evaluation mechanism enhances decision quality, while a local rescheduling post-processing module facilitates both localized adjustments and global insertion refinement of action sequences. Numerical experiments demonstrate that our method reduces total operational cost by 11.84%, 8.93%, 30.46%, and 7.24% compared with the Dynamic Reciprocating Strategy (DRS), Clustering Reciprocating Strategy (CRS), Google OR-Tools solver, and Basic DRQN in the illustrative experiment, and achieves an average cost reduction of 13.67% over DRS, 14.66% over CRS and 4.90% over Basic DRQN (p < 0.001) across 20 random test instances, validating the framework’s effectiveness and adaptability for solving complex three-dimensional spraying scheduling problems in numerical simulations.