Resource Allocation for Underwater Acoustic Sensor Networks With Partial Spectrum Sharing: When Optimization Meets Deep Reinforcement Learning

To utilize the limited acoustic spectrum while combating the harsh underwater propagation, we incorporate partial spectrum sharing into an underwater acoustic sensor network and aim to maximize the minimum data collection rate among all underwater sensor nodes through joint power allocation and spectrum assignment. To cope with the non-convex optimization problem, we propose a Hybrid Model-based and Data-based Resource Allocation (HMDRA) scheme: 1) Under any given spectrum assignment strategy, we analyze the impact of the partial spectrum sharing and imperfect successive interference cancellation on baseband signal processing, and formulate a power allocation problem that is solved by the bisection method and Lagrange dual theory. 2) Based on the optimal power allocation strategy, the gradient-free genetic algorithm (GA) is first adopted to approach the optimal solution of the model-less spectrum assignment problem by nearly enumerating the solution space. To reduce complexity, we further propose a deep reinforcement learning (DRL)-based algorithm and obtain an efficient solution by traversing a deep neural network-based policy learned from the training stage. Simulation results show that compared with the GA-based algorithm, the average execution time of the DRL-based algorithm is substantially reduced by 5 orders of magnitude to 0.7076 seconds at the cost of approximately 6 percent performance loss.