Optimizing Synchronization Times for Tracking a Mobile Asset in GPS-denied Environments using Deep Q-Learning

We consider a scenario in which a distributed sensor network is tasked with localizing a remote mobile asset operating in a GPS-denied environment. The remote asset transmits a beacon signal at known intervals, and the sensor network uses time-of-flight measurements to perform localization. However, the measurements are corrupted by noise in the clocks at the sensors and noise in RF reception. Moreover, the noise is nonstationary, as the sensors' clocks noise increases over time unless the sensors perform synchronization, and the RF signal-to-noise ratio (SNR) depends on the distance between the asset and each sensor, which varies with the location of the asset. The problem of determining the optimum synchronization times can be modeled as a partially observable Markov decision process (POMDP), and POMDPs are challenging to solve optimally. We apply deep Q networks (DQNs) to find near optimal synchronization times and solve for the asset location that minimizes the squared error. In comparison to our previous work, which quantized the belief space and applied tabular Q-learning, this paper considers a more general noise model and the use of a DQN learning algorithm that works with different SNRs across the sensors. The results show that DQNs can provide performance improvements over previously implemented methods, such as tabular Q-learning.