Toward Multi-Modal Semantic Communication in Reliable IoV: Joint Multi-Role Prompts and GenAI

The Internet of Vehicles (IoV) plays a pivotal role in advancing an intelligent and sustainable transportation ecosystem. However, the rapid surge in multi-modal data traffic, driven by the proliferation of diverse applications and services, has highlighted major challenges to traditional IoV communication methods, particularly concerning reliability and efficiency. In this article, we present a novel multi-modal semantic communication (Multi-MSC) scheme aimed at improving the reliability of multi-modal data transmission by integrating multi-role prompts and generative artificial intelligence (GenAI). Specifically, the multi-modal semantic encoder in the Multi-MSC transforms source data into core semantic text (CST) based on multi-role prompts, followed by adaptive channel coding to ensure reliable CST transmission in a high-interference wireless environment (HIWE). The recovered CST is then input into a high-quality scene generator powered by GenAI, addressing the diverse needs of various IoV applications. Additionally, the above components are semantically updated and aligned through the semantic knowledge base. Experimental results show that our method significantly improves the accuracy of multi-modal data transmission in various HIWEs and enables high-fidelity scene content generation with CLIP scores exceeding 0.73.