Reinforcement Learning Based Environment-Aware V2I Anti-Jamming Communications

The performance of reinforcement learning (RL)-based anti-jamming communication schemes in vehicular networks deteriorates in high-density traffic area with complex building layouts due to the spatio-temporally varying jamming signal strength and interference levels. In this paper, we propose an RL-based environment-aware vehicle-to-infrastructure (V2I) communication scheme to choose the transmit power and frequency channel, which exploits environmental features such as building layouts and traffic density in the state formulation to indicate the future channel gains, interference levels and jamming signal strength. This scheme enhances the utility as a weighted sum of signal-to-interference-plus-noise rate, communication energy consumption, frequency-hopping costs, and a penalty factor for dissatisfying quality-of-service requirements to avoid choosing risky V2I communication policies. The movement trajectories of vehicles, parameter of vehicular channel model and environmental features are used to formulate the simulated V2I communication experiences for updating Q-values in each time slot to accelerate learning speed. A deep version is further proposed to extract features of radio propagation in the area with variable traffic density and complex building layouts, as well as to address the quantization errors of the jamming signal strength, channel gain and interference levels in the RL state formulation, thereby improving communication reliability and energy efficiency. Experimental results based on three vehicles driving in campus area show the performance gains of transmission latency and communication energy consumption compared with the benchmark against a smart jammer.