F <scp>ed</scp> D <scp>i</scp> AL: Adaptive Federated Learning with Hierarchical Discriminative Network for Large Pre-trained Models

Large pre-trained models have significantly advanced computer vision (CV) and natural language processing (NLP). However, their deployment is challenging in privacy-sensitive scenarios where data must remain decentralized. Federated Learning (FL) addresses this issue by enabling local model training without sharing raw data. Despite this advantage, integrating large models into FL introduces challenges such as training inefficiencies, label scarcity, and data heterogeneity. In this paper, we propose FedDiAL, a framework designed to effectively incorporate large pre-trained models into FL while addressing these challenges. At its core, FedDiAL, features the novel HDis-Net, which enhances the efficiency of large models in resource-constrained environments. We introduce a two-phase training strategy with probabilistic feature augmentation to improve feature discrimination. Additionally, we propose an adaptive pseudo-labeling method to generate high-confidence labels, mitigating label scarcity. To handle data heterogeneity, we develop a focused fine-tuning strategy that adapts HDis-Net to diverse client data distributions. Our theoretical analysis establishes the convergence of HDis-Net. Extensive experiments on four datasets, including Tiny ImageNet and AG News, demonstrate that FedDiAL outperforms state-of-the-art methods, achieving up to a 35.66% accuracy improvement in both CV and NLP tasks.