A Survey on Vision-Language Models for Multimodal Federated Learning Tasks

Multimodal Federated Learning (MMFL) enables privacy-preserving training on decentralized visual-text data, facilitating collaborative learning across clients with heterogeneous modalities. Effectively modeling multimodal heterogeneity in such settings is crucial. Vision-Language Models (VLMs), renowned for their powerful visual-text alignment capabilities, emerge as promising solutions for enhancing MMFL. However, deploying large-scale VLMs in decentralized MMFL environments presents significant integration challenges, ranging from misaligned modality semantics to client-specific constraints, extending beyond traditional federated optimization issues. This survey presents the first comprehensive review of VLM-based MMFL approaches. Specifically, we: (i) outline the foundational concepts of existing VLMs and MMFL frameworks; (ii) categorize current research into four representative tasks: cross-modal content generation, Visual Question Answering (VQA), multimodal retrieval, and Multimodal Sentiment Analysis (MSA); (iii) highlight the trustworthiness including security, robustness, fairness, and interpretability, which underpins the design and evaluation of VLM-based MMFL systems; (iv) explore key scientific challenges in MMFL research, including high communication overhead, cross-client semantic drift, distributed VLMs inference, multimodal fairness and bias, and privacy-fusion trade-offs; and (v) conclude by outlining open research directions, emphasizing the development of lightweight, communication-efficient architectures, privacy-aware multimodal alignment, fairness-aware training objectives, and robust methods for handling non-IID multimodal data distributions.