Multi-Agent Reinforcement Learning Based Adaptive Parameter Optimization for Semi-Persistent Scheduling in C- V2X Mode 4

The Third Generation Partnership Project (3GPP) has standardized cellular vehicle-to-everything (C- V2X) Mode 4 to facilitate direct communication between intelligent connected vehicles. In Mode 4, vehicles autonomously reserve and select wireless spectrum resources through sensing-based semi-persistent scheduling (SPS). However, the half-duplex transmissions and hidden terminal problems in SPS could degrade the quality of services (QoS), possibly making it hard to meet the requirements for basic safety services. To support the SPS algorithm's performance in highly congested conditions, this paper presents QMIX-SPS, an adaptive parameter optimization methodology. It also proposes a new performance metric, the signal-to-interference-plus-noise ratio (SINR) achievement rate, as a means of evaluating communication network effectiveness. We developed a multi-agent vehicular communication framework in which parameters of the SPS, including transmission power, resource reservation probabilities, resource reservation counter-offset steps, and candidate resource ratios, are periodically adjusted under the guidance of the QMIX reinforcement learning algorithm. Our proposed resource allocation algorithm utilizes the mechanism of value decomposition to solve the reward allocation problem when competition and collaboration coexist in a V2X environment. Simulation results show that QMIX-SPS outperforms the baseline algorithm. Furthermore, the algorithm has excellent stability flexibility, and compatibility with the SPS.