Environment-Aware Beam Probing Codebook Design for Two-Timescale Beam Alignment in RIS-Assisted Communication Systems

Reconfigurable intelligent surfaces (RISs) have paved the way for addressing the blockage issue via reshaping the propagation environment. However, given the passive nature of RIS and the mobility of users, optimal directional beam pairs among base station (BS), RIS and users are rather difficult to obtain for the sake of high-quality data transmission. Herein, in this letter, we propose a two-timescale deep reinforcement learning (DRL) based beam alignment method for real-time optimal directional beam selection using small-size probing codebooks. Thereof, the learned probing codebooks are in accord with the users' environmental characteristics. Furthermore, by exploiting probing feedback, the double convolution gate linear units (C<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula>-GLU) are introduced to realize realistic feature extraction in order for the optimal beam pair matching. The results demonstrate that the proposed method obtain better performance compared to benchmark methods.