KGE-MNER: Knowledge Graph Enhanced Multimodal Named Entity Recognition

Multimodal Named Entity Recognition (MNER) has emerged as a crucial research direction, bridging Natural Language Processing (NLP) and Computer Vision (CV). MNER aims to accurately recognize entities across different modalities. The current approaches to Named Entity Recognition (NER) primarily model the internal matching relationships between pairs of images and text independently. Its often leads to the under-exploration of entities with weaker relationships. For this problem, we propose a Knowledge Graph Enhanced Multimodal Named Entity Recognition method (KGE-MNER). This approach use the CLIP model to identify entities from both text and image modules. Then we use the relationships between entities within an external knowledge graph to construct precise entity pairs. This enhances the accuracy of NER, which is essential for downstream tasks like relation extraction and entity linking. Experimental results show that our approach greatly enhances the efficiency and accuracy of entity recognition (ER).