Beyond Isolated Features: Group-Level Feature-Driven Multimodal Fusion for Entity Relationship Extraction

Named entity recognition and relation extraction are two crucial techniques in the construction of knowledge graphs, as their performance directly impacts downstream tasks. In scenarios such as social media, where text is short and contains numerous references, relying solely on a single modality often leads to suboptimal entity-relation extraction results. To address this issue, this paper proposes a multimodal entity-relation extraction model. The model incorporates group-level features into visual representations and integrates them into a BERT variant using a dynamic gating mechanism and an attention mechanism. This approach effectively reduces modal noise, balances the gap between modalities to some extent, and enhances both encoding performance and data utilization. The experimental results demonstrate that the proposed model achieves F1 scores of 87.78% and 84.16% on public datasets for MNER and MRE tasks, respectively, outperforming baseline models. Additionally, ablation studies validate the effectiveness of each module within the proposed model.