Privacy-Preserving LLM-based RAG with Split Inference and Masked Privacy Recovery

With the development of 6G endogenous intelligence, retrieval-augmented generation (RAG) based on large language models (LLMs) becomes one of its key technologies due to its ability to empower 6G intelligent systems with enhanced knowledge reserves and context-aware decision-making capabilities. However, deploying LLMs on cloud servers poses privacy risks to users and businesses. To address this, we propose a privacy-enhanced retrieval-augmented generation framework (PE-RAG). It uses split inference by deploying the embedding and output layers of LLMs at the edge, preventing sensitive information in plain text from being stolen by attackers on untrusted cloud servers. In addition, we propose the privacy masking recovering knowledge base (PMR-KB), which masks privacy sensitive content in the knowledge base and recovers masked content in the inference results, thereby minimizing utility degradation while reducing privacy leakage risks. Our experiments show that the proposed framework achieves significant improvements over state-of-the-art approaches, reducing privacy leakage by $76.4 \%$ and enhancing inference utility by $80.0 \%$.