Cooperative Reinforcement Learning for Energy Management in Multi-Hop Networks With Energy Harvesting

This paper proposes a Reinforcement Learning (RL) framework for joint transmit-sleep scheduling for multi-hops wireless sensor and IoT networks with energy harvesting. The goal of the framework is to learn situation-specific scheduling policies for reducing energy expenditure, while maintaining network performance in terms of packet delivery ratio and end-to-end delay. The proposed system uses a cooperative RL approach where two learning agents, deployed per node, jointly learn transmit and sleep scheduling strategies to manage network energy budgets. The RL framework also uses a localized learning confidence parameter sharing strategy that allows the nodes to ignore unreliable RL observations. This makes the system scalable for network topologies where the source and destination nodes are separated by many numbers of hops. The learning module is decentralized in that learning is independently carried out at the sensor/IoT nodes without relying on a central learning coordinator. Decentralized learning makes the system computationally efficient, and also avoids energy and communication bandwidth overhead usually imposed by a central coordinator. With simulation studies, the proposed learning-driven protocol is tested and compared against existing known MAC sleep schedulers. The proposed mechanism is validated for different scenarios with heterogeneous topologies, traffic, and energy harvesting conditions.