Cooperative Jamming for Secure Air–Ground Integrated Networks: A Hierarchical Distributed Deep Reinforcement Learning Approach

Recently, Internet of Things (IoT) devices have been installed everywhere, and these devices are connected through wireless communication networks. In particular, unmanned aerial vehicles (UAVs), one of the most promising IoT devices, are expected to be used actively in Internet of Battlefield-Things (IoBT) networks due to their flexible three-dimensional (3D) mobility. To react to enemy UAV attacks in the IoBT networks, the ground-to-air (G2A) or air-to-air (A2A) radio jamming can be an effective counterattack technique that disrupts the communication and control signals of adversary equipment. That is, it can be a very effective means of coping with attacks by UAVs in modern battlefields characterized by electronic warfare. Accordingly, this paper proposes a hierarchical distributed deep reinforcement learning-based cooperative jamming (HDRL-CJ) method for secure air-ground integrated networks. The proposed method uses two types of jammers: ground jammers (GJ) and UAV jammers (UJ). The GJ optimizes the beamwidth to maximize the effectiveness of jamming, and the UJ tracks the malicious UAV (MU) and controls the jamming power to minimize the MU’s signal-to-jamming-plus-noise ratio (SJNR) while considering the UJ’s limited battery capacity. Moreover, to reduce the computational complexity of reinforcement learning (RL) method, we devise a hierarchical RL architecture that separates the UJ’s movement control and transmit power control. Through extensive simulations, we demonstrate that the proposed HDRL-CJ method converges to the optimal solution obtained by the optimal exhaustive search algorithm. Furthermore, by comparing the jamming performance with several benchmark methods, we validate the proposed method’s superior performance under various 3D network environments.