HGFF: A Deep Reinforcement Learning Framework for Lifetime Maximization in Wireless Sensor Networks

Planning the movement of the sink to maximize the lifetime in wireless sensor networks (WSNs) is an essential problem. Many existing mobile sink techniques based on mathematical programming or heuristics have demonstrated the feasibility of the task. Nevertheless, the huge computational cost or the over-reliance on human knowledge can result in relatively low performance. To balance the need for high-quality solutions to minimize inference time, we propose a new framework to construct the movement path of the sink automatically. We cast the lifetime maximization problem as an optimization task within a heterogeneous graph and learn movement policy for the sink by combining graph neural network (GNN) with deep reinforcement learning. Our approach comprises three key modules: 1) a heterogeneous GNN to learn representations of sites and sensors by aggregating features of neighbor nodes and extracting hierarchical graph features; 2) a multihead attention mechanism that allows the sites to attend to information from sensor nodes, which highly improves the expressive capacity of the learning model; and 3) a greedy policy that learns to append the next best site in the solution incrementally. We design twelve types of static and dynamic maps to simulate different WSNs in the real world, and extensive experiments are conducted to evaluate and analyze our approach. The empirical results show that our approach consistently outperforms the existing methods on all types of maps. Notably, our approach significantly extends the simulation lifetime without sacrificing a large increase in inference time.