Towards Privacy-Preserving Code Generation: Understanding and Mitigating Memorization in Code Large Language Models

Recent advances in code-specialized language models (LLM4Code) have revolutionized software development by assisting with tasks like code generation, debugging, and testing. However, training these models on large, centralized datasets raises privacy concerns, as memorization of training data can lead to confidentiality and intellectual property risks. As a result, such models are often trained on publicly available data. This research examines code memorization patterns and explores privacy-preserving techniques, such as Federated Learning and Differential Privacy, to mitigate these risks. The goal is to develop secure, high-performance, and inclusive models that protect sensitive information, foster trust, and enable broader adoption in software engineering.