A MAB-Based Two-Stage Beamforming and Combining for Air-to-Air mmWave Communication

In this paper, we propose a two-stage beamforming and combining (TSBC) scheme based on multi-armed bandit (MAB) in an energy-constrained air-to-air mmWave MIMO communication. Different from existing schemes, we use the MAB to design the pre-beamforming matrix and the pre-combining matrix from historical data, such that the dimension of the effective channel is reduced, thereby reducing the pilot and improving the spectrum efficiency (SE) and energy efficiency (EE). Particularly, in the first stage, the joint design problem of the pre-beamforming matrix and pre-combining matrix for the transmitting and receiving unmanned aerial vehicles is formulated as a budgeted combinatorial MAB (BC-MAB) problem. In this formulation, the codeword, received energy, battery capacity, and energy consumption are associated with the arm, reward, budget, and cost, respectively. To simplify the solution to the BC-MAB problem, its dual problem is formulated. We propose a two dimensional upper confidence bound with bangperbuck ratio (2D-UCBBR) algorithm that leverages the learning capabilities of MAB to address this dual problem, which does not require channel state information. We then prove the convergence of the 2D-UCBBR algorithm. In the second stage, the effective channel matrix is estimated and used to design the beamforming and the combining matrices to increase the SE. Due to the limited numbers of activated transmitter and receiver RF chains, the dimensions of pre-beamforming and pre-combining matrices are reduced, yielding a low-dimensional effective channel matrix and a low pilot overhead, thereby improving both SE and EE. Simulation results validate the good performance of the proposed scheme.