Semantic Communication-Aware Federated Fine-Tuning of Large Language Models

Fine-tuning large language models (LLMs) in federated learning (FL) environments faces significant challenges due to constrained client resources and high communication costs, limiting the scalability and feasibility of decentralized LLM adaptation. To address this problem, we present a novel approach for semantic communication that goes beyond model partitioning-based training by adding a semantic autograd bridge (SAB). In our approach, during forward and backward propagation, SAB decodes intermediate tensors into low-dimensional semantic representations instead of sending whole activations and gradients across clients. We also incorporate a compression-sensitive client selection technique to improve communication latency under bandwidth and memory limitations. As a result, the system dynamically distributes model partitions to the most qualified clients, accelerating training, and ensuring stable convergence across heterogeneous devices, allowing rapid and scalable federated fine-tuning of LLMs. Experiments on WikiText-2 and AG News reveal that semantic communication can significantly improve the scalability of decentralized LLM training, reducing communication volume by over 90% while maintaining accuracy within 1% of centralized baselines.