Multimodal Knowledge Graph Vision-Language Models for Precision Retrieval and Inference in Embedded Systems

In light of the challenges associated with enhancing multimodal knowledge graph retrieval and improving embodied intelligence in dynamic environments, we propose the Multimodal Knowledge Graph Vision-Language System (MKGVL). This system is designed to enhance the reasoning and feedback capabilities of embodied intelligent systems by continuously updating the knowledge graph through real-time feedback from the Vision-Language Model (VLM). By integrating visual encoders, language models, and knowledge graph networks, MKGVL constructs a unified multimodal representation that enables adaptive decision-making and responsiveness to environmental changes. Experimental results show that MKGVL outperforms existing models in fine-grained retrieval tasks, achieving an 11.5% improvement in Rank1 accuracy and a mean Average Precision (mAP) of 97.49%. Further evaluations conducted on datasets such as ARKitScenes, MultiScan, and 3RScan highlight the model's robustness and adaptability. Additionally, MKGVL's deployment on embedded platforms like Jetson Orin demonstrates its efficiency in real-time multimodal tasks, particularly in resource-constrained environments. These findings underscore MKGVL's ability to deliver accurate and efficient multimodal processing, making it a strong solution for adaptive knowledge-based systems in complex settings.