Semantic-Aware Beam Prediction Approach with Multimodal Fusion

Beam prediction is crucial to reduce the beam training overhead of large-scale antenna array systems. This paper proposes a semantic-aware beam prediction scheme based on multimodal fusion. Specifically, we develop a network architecture that integrates the semantic-aware network, encoder, multimodal fusion module, and decoder. The architecture takes environment images captured at the base station (BS) and coordinates text collected from GPS sensors equipped on targets as input, training the network to predict the optimal beam index. The semantic-aware network preprocesses the environment images captured at the BS, extracting key semantic images relevant to communication targets and discarding redundant environmental semantics to reduce data storage and computational complexity. In the multimodal fusion module, we employ a multimodal compact bilinear pooling (MCBP) method to reduce the complexity of outer product calculations, mapping the highdimensional results of the outer product to a lower-dimensional space for a more compact and efficient representation. Simulation results indicate that the beam prediction scheme proposed in this paper can achieve satisfactory accuracy without the need for pilot overhead and beam measurement and efficient prediction with low computational complexity.