Adaptive PCI Allocation in Heterogeneous Networks: A DRL-Driven Framework With Hash Table, FAGA, and Guiding Policies

The proliferation of 5G heterogeneous networks has significantly increased the number of base stations and user equipments (UEs), leading to heightened wireless interference, network latency, and reduced reliability. Efficient PCI (Physical Cell Identity) allocation is essential for resolving conflicts and confusions, thereby improving resource utilization and network performance. This paper presents an intelligent DRL-based framework for adaptive PCI allocation in large-scale heterogeneous networks. The proposed approach integrates Multi-Objective Simulated Annealing (MOSA) to resolve diverse PCI conflicts. The combination of Deep Deterministic Policy Gradient (DDPG) with an experience pool enhances decision-making efficiency and training stability, while a Fitness-Aided Genetic Algorithm (FAGA) dynamically adjusts key parameters to ensure rapid convergence. A Hash Table supports efficient initial PCI allocation, reducing early-stage conflicts, and a guiding policy accelerates convergence through targeted Q-value calculation. In simulations, standard DRL converged to a loss of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {10^{-3}}$ </tex-math></inline-formula>, while the proposed model, integrating MOSA, Hash Table, and FAGA, reduced PCI conflict resolution time and achieved more efficient convergence to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {10^{-5}}$ </tex-math></inline-formula>. Specifically, MOSA improved performance by 87.86%, with Hash Table and FAGA contributing gains of 63.10% and 59.94%, respectively, in optimizing training efficiency. These results highlight the model’s adaptability, generalization capability, and practical implications for improving communication network performance.