SLDP-LoRA: A Privacy-Preserving Split Learning Framework With Low-Rank Adaptation

The advancement of Large Language Models has revolutionized natural language understanding, offering powerful capabilities. However, deploying LLMs in privacy-sensitive environments, such as Internet of Things (IoT) systems, presents substantial challenges to ensuring user data confidentiality. This paper introduces SLDP-LoRA, a novel privacy-preserving split learning framework that combines Rényi Differential Privacy with Low-Rank Adaptation for efficient and secure fine-tuning. SLDP-LoRA minimizes computational overhead on user devices by performing token representation and privacy perturbation locally, while employing LoRA-based fine-tuning and denoising techniques on the server side. The framework ensures strong privacy protection through dynamic noise injection tailored to token representations, maintaining high utility. Experimental results on multiple datasets and models demonstrate SLDP-LoRA's superior performance, with an average improvement of 25.55% in accuracy and a 74.9% reduction in privacy leakage compared to the state-of-the-art methods. Ablation studies further validate the effectiveness of its components in achieving a robust utility-privacy trade-off. SLDP-LoRA offers a scalable, efficient, and practical solution for privacy-preserving applications in distributed and resource-constrained environments.