Predictive ISAC Beamforming Using LoRA-Transformer LSTM Integrated With GNN in Ultra-Dense D2D mmWave Networks

Effective beamforming is crucial in ultra-dense device-to-device (D2D) mmWave networks to reduce interference and optimize communication. However, current methods often struggle to balance high data rates with minimal signaling overhead, especially with imperfect channel state information (CSI). This paper introduces a machine learning-assisted beamforming design tailored for integrated sensing and communication (ISAC), combining an efficient Low-Rank Adaptation Blocked LSTM Transformer with Message Passing Graph Neural Network that is called as <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoRA_BLT-MPGNN</monospace>. The <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoRA_BLT</monospace> component provides a key advantage by efficiently capturing both temporal and spatial patterns through fixed-weight updates during training and low-rank approximation for fine-tuning, significantly reducing computational complexity without sacrificing accuracy. Using historical channel data, the method implicitly captures channel characteristics and directly predicts the beamforming matrix for the subsequent time slot, minimizing signaling overhead while maximizing the achievable sum rate. Our research and simulation results show that the proposed predictive beamforming design maintains robust sensing performance and achieves a sum rate closely approaching the upper bound achievable with perfect CSI. Additionally, the incorporation of MPGNNs enables efficient modeling and learning of complex inter-node relationships, making the approach adaptable to networks of varying sizes and densities. Simulations further demonstrate that the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoRA_BLT-MPGNN</monospace> model can be generalized across different network densities and coverage areas, reducing computational complexity by approximately <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$32\times$</tex-math></inline-formula> in terms of Tensor Floating Point Operations (TFLOPs) compared to conventional methods while maintaining high performance.