VRCMC: a semi-supervised method for multi-modal entity alignment with visual refinement and cross-modal calibration

The goal of multi-modal entity alignment (MMEA) is to automatically discover entities across different multi-modal knowledge graphs (MMKGs) that refer to the same real-world object. However, because of the complexity and diversity of data sources, entities in MMKGs often lack images or possess multiple redundant images, which limits alignment accuracy. The multi-modal feature extraction and fusion process of existing MMEA methods fails to effectively filter out noise in visual information. In addition, the iterative learning strategies employed by these methods rely on a single representation to generate pseudo labels, which are originally designed for single-modal entity alignment and are inadequate for multi-modal application scenarios. In this paper, we propose a novel semi-supervised multi-modal entity alignment approach based on visual refinement and cross-modal calibration (VRCMC) to address the above problems. Specifically, it first leverages pre-trained, high-quality structural features to refine the noisy visual features. Subsequently, the fusion of different modal features is dynamically adjusted at the entity level through a progressive freezing and unfreezing mechanism. Finally, a cross-modal calibration module composed of multi-modal voting and modality reweighting is put forward to obtain reliable pseudo labels during iterative learning for multi-modal entity alignment. Comprehensive experimental evaluations on five real-world datasets show that our method consistently outperforms existing MMEA approaches and achieves state-of-the-art performance.