Distributed Time-Slotted Channel Hopping in IoT Networks Using Multi-Agent Bernoulli Bandits

The proliferation of Internet-of-Things (IoT) devices has led to significant challenges in managing radio access and energy efficiency in resource-constrained wireless sensor networks (WSNs). The Time-Slotted Channel Hopping (TSCH) protocol, part of the IEEE 802.15.4e standard, offers a promising solution for addressing these challenges, particularly through its collision-free scheduling mechanism. However, existing TSCH scheduling schemes often rely on centralized coordination or extensive control message exchanges, leading to inefficiencies in terms of energy consumption and bandwidth usage. We propose a novel decentralized TSCH scheduling framework driven by Bernoulli Multi-Armed Bandit (MAB) learning. Our approach enables each node to independently learn and optimize its transmission schedule without requiring control information from a central server or neighboring nodes. We explore multiple bandit action selection policies, including <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-greedy, Upper Confidence Bound (UCB), and Thompson Sampling, and highlight their limitations in low data-rate networks. To overcome these challenges, we introduce a Low-Rate Resilient Policy (LRRP) that enhances TSCH scheduling in sparse traffic conditions by synthesizing packets to compensate for sample deficiencies. Experimental results demonstrate that our framework achieves collision-free scheduling and substantial energy savings while ensuring scalability for large networks. The proposed method outperforms traditional policies, particularly in heterogeneous and low-traffic environments, making it highly suitable for resource-constrained IoT networks. The bandit-driven scheduling approach is shown to achieve upto <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\approx 19\%$ </tex-math></inline-formula> increase in throughput for a 50-nodes IoT network, compared with existing scheduling mechanism.