VSGN: Visual–Semantic Guided Interaction Network for Multimodal Named Entity Recognition

Multimodal Named Entity Recognition (MNER) aims to integrate textual and visual information to identify entities with specific semantic categories. However, existing methods often suffer from insufficient intra-modal semantic modeling, coarse cross-modal alignment, and vulnerability to noisy or ambiguous expressions in social media. To address these challenges, we propose a Visual–Semantic Guided Interaction Network (VSGN), which improves multimodal representation learning from both semantic and structural perspectives. Specifically, we first design an adaptive visual–semantic fusion module that incorporates visual descriptions as semantic guidance, enabling more informative cross-modal interactions. To further enhance feature quality, we introduce a deviation-aware channel-wise inhibitory routing (CIR) mechanism, which jointly models channel importance and distributional deviation to suppress noisy or redundant visual signals. In addition, we propose a visual–semantic guided graph structure learning module (VSG), which explicitly captures structural dependencies across modalities. By enforcing distribution-level alignment between textual and visual graph representations, the model achieves structure-aware cross-modal interaction and reduces modality inconsistency. Extensive experiments on the Twitter-2015 and Twitter-2017 datasets demonstrate the effectiveness of the proposed method, achieving F1 scores of 76.72% and 87.86%, respectively. The results show that jointly modeling semantic enhancement and structural alignment leads to more robust and discriminative multimodal representations.