Multimodal Semantic Communication: Research Progress, Key Challenges, and Future Trends

With the rapid advancement of AI, deep learning has been increasingly integrated into communication systems, driving the evolution of semantic communication. By extracting and transmitting task-relevant semantics from raw signals, this paradigm enhances efficiency while alleviating bandwidth and redundancy constraints. Multimodal data, as a dominant representation form, has shown promising results in tasks like VQA and cross-modal retrieval. However, its application in semantic communication remains nascent, with most studies limited to unimodal scenarios. Challenges persist in achieving modality-coordinated compression, cross-modal semantic alignment (the process of ensuring consistency between transmitted and received semantics, or aligning representations across modalities), and robust task-oriented transmission under channel variations. Addressing these issues necessitates the development of a unified multimodal semantic communication (MMSC) framework. Such efforts are crucial for enabling intelligent communication in future systems, including 6G, the metaverse, and IoV. This article reviews recent progress in MMSC, outlines key challenges, and explores future research directions in this emerging field.