A multi-armed bandits empowered transmission scheme for IRS-assisted MISO system

Intelligent reflecting surface (IRS) is an effective technology that can enhance millimeter-wave communication performance and address transmission challenges. However, conventional methods require accurate channel state information when jointly optimizing precoding and IRS phase shifting, and pilot consumption increases with the number of base station antennas and IRS components. To reduce pilot overhead, a transmission design scheme based on multi-armed bandit (MAB) was proposed. Specifically, in IRS-assisted single antenna user downlink communication systems, the problem of IRS phase shift design and precoding beamforming was decoupled through a dual time scale approach. Using the contextual MAB algorithm in reinforcement learning, the IRS phase shift matrix was designed within each coherent time block based on contextual information and reward feedback. Next, the zero forcing method was used to design precoding, and the greedy method was introduced to select actions at each moment to avoid getting stuck in local optima. The simulation results show that the proposed algorithm can achieve higher effective spectral efficiency than the existing algorithms.