Addressing the Curse of Scenario and Task Generalization in AI-6G: A Multi-Modal Paradigm

Existing works on machine learning (ML)-empowered wireless communication primarily focus on monolithic scenarios and single tasks. However, with the blooming growth of communication task classes coupled with various task requirements in future 6G systems, this working pattern is obviously unsustainable. Therefore, identifying a groundbreaking paradigm that enables a universal model to solve multiple tasks in the physical layer within diverse scenarios is crucial for future system evolution. This paper aims to fundamentally address the curse of ML model generalization across diverse scenarios and tasks by unleashing multi-modal feature integration capabilities in future systems. Given the universality of electromagnetic propagation theory, the communication process is determined by the scattering environment, which can be more comprehensively characterized by cross-modal perception, thus providing sufficient information for all communication tasks across varied environments. This fact motivates us to propose a transformative two-stage multi-modal pre-training and downstream task adaptation paradigm. In the pre-training stage, we introduce a multi-modal two-tower model and a corresponding contrastive learning method to integrate the explicit description of the scattering environment and implicit channel state information (CSI) into a universal representation, which encapsulates rich high-level knowledge and can be leveraged for all downstream tasks in different scenarios. Additionally, we present two specially designed model structures to enhance the interaction of communication modalities. In the second stage, based on the frozen pre-trained model, we propose a direct method and a pluggable method for flexible and low-cost task adaptation. Experimental results demonstrate that our proposed approach significantly outperforms benchmarks in both task performance and tuning parameter size for exemplary sub-tasks in unseen scenarios.