LLM-Enabled Multi-Modal Data Synthesis via Cross-Domain Collaboration

Data is pivotal to the advancement of intelligent communication and network systems. However, the availability of relevant data is hampered by different challenges, including scarcity, privacy issues, and the high cost of acquisition. Furthermore, existing data generation models are of low quality, uncontrollable, and lack generalizability. In this article, we propose a large language models (LLMs) driven framework that leverages the power of LLMs in concert with various domain-specific generative models (DGMs) to realize general multi-modal data synthesis. Specifically, our method employs LLMs as the core to interpret user requests, decompose a complex task into a manageable set of sub-tasks, and delegate each sub-task to the most suitable DGM, thereby automatically constructing customized data generation pipelines. Meanwhile, DGMs contribute their expertise to generate high-fidelity, domain-relevant data, whose specialized knowledge can be further enhanced by the LLM's broad linguistic knowledge via knowledge transfer. In addition, the integration of reinforcement learning (RL) is promising to enhance the framework's ability to optimally utilize DGMs, resulting in data generation with superior quality and control flexibility. Experimental results demonstrate the effectiveness of LLMs in augmenting domain-specific generative models via knowledge transfer and in facilitating multi-modal data synthesis through collaborative interactions with diverse DGMs.