The More You Tune, the More They Spill?

Abstract Although previous research has established that pre-trained code models can regurgitate their training data via extraction attacks, the privacy implications of the increasingly common practice of fine-tuning remain underexplored. Fine-tuning enables entities to adapt models to specialised tasks using smaller, curated datasets that may contain proprietary assets or sensitive information. This study provides a comparative investigation into data extractability, before and after this fine-tuning stage. We develop a benchmark to systematically assess the vulnerability of pre-training and fine-tuning samples to extraction attacks. Our experiments on the Starcoder2 models yield two key findings. First, fine-tuning significantly reduces the extractability of the model’s original pre-training data, decreasing the extraction rate from 54.9% to 23.5%. Second, this benefit is offset by the increased vulnerability in the fine-tuning data. Smaller models, in particular, show a strong propensity to leak this data. Extending this analysis, we find that moderate model compression generally preserves the same memorization patterns, whereas int4 quantization reduces absolute pre-training memorization but amplifies relative fine-tuning leakage. We further find that parameter-efficient fine-tuning mitigates fine-tuning data memorization compared to full fine-tuning, while exhibiting smoother forgetting of pre-training data. A qualitative analysis of 2,000 samples reveals that data structures and licensing information are the most frequently memorized categories. Our work highlights that fine-tuning introduces a privacy trade-off, shifting the risk from vast public pre-training data to smaller, higher-value proprietary fine-tuning data. Model deployers should evaluate these risks and explore strategies to mitigate leakage while preserving performance.