IBTD: A novel ISAC beam tracking based on deep reinforcement learning for mmWave V2V networks

Abstract Beam tracking is commonly employed in millimetre wave (mmWave) based vehicle‐to‐vehicle (V2V) networks to align the beams towards the intended targets and compensate for the path loss of mmWave signals. To mitigate the high latency issue arising from the tracking processes, integrated sensing and communication (ISAC) technology leverages the echo signal to sense the motion parameters of the target, achieving low‐latency beam tracking without requiring pilot and uplink feedback. Existing studies mainly focus on utilizing ISAC for beam alignment to track the target, without integrating beam tracking with resource allocation. In this paper, we propose the ISAC beam tracking based on deep reinforcement learning (IBTD) algorithm to address this problem. Specifically, we introduce the concept of packet age to measure communication performance. To achieve accurate beam tracking and optimize the transmit power, we integrate the sensing results, such as the position and velocity of the target vehicle, along with the buffer pool status information, with deep reinforcement learning (DRL) to select an appropriate policy. Furthermore, we consider the effect of inter‐vehicle distance and incorporate the changing of tracking targets into the DRL‐based policy. Simulation results demonstrate that the proposed IBTD algorithm achieves lower packet age and transmit power consumption compared to the baseline algorithms.