Reinforcement Learning Based Covert Routing with Node Failure Resiliency for Heterogeneous Networks

Due to the ever-increasing demand for wireless networks with improved resilience, coverage, and throughput, heterogeneous networks (HetNets) have been widely investigated for relay networks equipped with multiple communications technologies. Despite these efforts, there are key challenges such as meeting covertness requirements to mitigate detection by potential adversaries and ensuring resilience and adaptation in the presence of network dynamics. In this paper, we propose a novel failure-adaptive covert routing technique using Q-learning for HetNets to ensure fast adaptation when relay nodes suddenly fail with some probability while maximizing covertness against the adversary. Our proposed Covert HetNet routing has three key components: 1) adaptive exploration, 2) exploiting existing Q-values incorporating the dynamics, and 3) adjusting Q-learning parameters. During the performance evaluations, we average over randomly selected topologies and consider different probability of the relay nodes failing to show that the proposed failure-adaptive covert routing ensures fast adaptation to node failures for various network topologies. Specifically, we show that using our proposed Covert HetNet routing achieves 31 times faster convergence than Q-learning using the ϵ-greedy method when nodes fail while achieving close to the maximum possible detection error probability at the adversary.