GAMBIT: A Generalized Adaptive Multi-Armed Bandit for Intelligent Beam Tracking and Rate Adaptation

Beamforming provides transmission/reception directivity gains that compensate for the high propagation loss encountered at millimeter-wave (mmWave) and sub-THz bands. However, narrow beams introduce significant beam misalignment challenges. Providing fast and efficient beam tracking is vital for maintaining communications and minimizing service disruptions. This paper introduces <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GAMBIT</i>, a restless multi-armed bandit (MAB) scheme for beam tracking and rate adaptation in mobile directional systems. GAMBIT aims to select the optimal beam and modulation and coding scheme (MCS) for upcoming transmissions through an online reinforcement learning technique called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Top-K Adaptive Thompson Sampling</i> (Top-K-ATS). According to this technique, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> beams (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K \geq 1$</tex-math></inline-formula>) are chosen and ranked based on previously estimated beam quality information. This information is initially gathered during cell discovery and is updated periodically based on explicit or implicit feedback from user equipment (UE). The best of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> selected beams, called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">“leader,”</i> is used for communications. The remaining beams, referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">“scouts,”</i> provide contextual information about the RF environment. To prevent beam quality information from becoming stale due to channel/mobility dynamics, we use beam coherence time analysis to derive an upper bound on the time between consecutive beam selection instances. We evaluate the performance of GAMBIT through simulations at 28 GHz and over-the-air (OTA) measurements at 28 GHz and 130 GHz. We compare our scheme with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula>-greedy, upper confidence bound (UCB), and Thompson sampling (TS) beam tracking algorithms. Results indicate that GAMBIT outperforms its contenders in both achievable data rate and outage probability.