Collaborative Reinforcement Learning for 3D UAV Localization Optimization Against GPS Spoofing

In this paper, the problem of using active unmanned aerial vehicles (UAVs) and a base station (BS) to jointly localize a target UAV under global positioning system (GPS) spoofing attacks is studied. In the considered model, active UAVs transmit signals, which will be reflected by the target UAV and received by active UAVs. Based on the signal transmission time, active UAVs calculate the distance between the target UAV and active UAVs. Then, active UAVs transmit these distance measurement information and their GPS information to the BS for localizing the target UAV. During the localization process, the target UAV is equipped with a GPS jammer, which can interfere with GPS information of active UAVs. Since the localization accuracy depends on distance between the target UAV and active UAVs and GPS information accuracy of active UAVs, active UAVs must optimize their trajectories and determine whether to transmit measurement information to the BS for localizing the target UAV. This problem is formulated as an optimization problem whose goal is to minimize the positioning error of the target UAV between the estimated and true positions of the target UAV by jointly optimizing the trajectories of active UAVs and determining distance information transmission scheme. To find the optimal solution, a historical observations and actions-based reinforcement learning (HOA-RL) method is proposed. Compared to traditional state-based reinforcement learning (RL) methods, the proposed method can capture the historical decision-making process of agents and optimally adjust trajectories of active UAVs and measurement information transmission scheme based on their observations. Simulation results show that the proposed method can achieve 44.4% and 70.4% gains in terms of reducing the positioning error of the target UAV compared to Qmix method and Qtran method, respectively.