Watermarking Fine-Tuning Datasets for Robust Provenance

Large Language Models are often fine-tuned on proprietary corpora, motivating reliable provenance signals. A corpus-level watermark method is proposed for fine-tuning datasets that survives training and common text transformations. The method subtly biases synonym choices according to a secret key (PRF) and encodes a multi-bit payload with an error-correcting code, enabling keyed detection via a generalized likelihood ratio test with permutation-calibrated p-values. For short offline passages (~100 words), the channel is valid but statistically underpowered: the average density is ~0.0165, and the median p-value is close to 1.0. In generative tests with Mistral 7B across 12 configurations and 12,720 texts, 0.00% detection was observed at very high quality (~99.8%). As limited base cases, positive detection was reported for other setups: 8.9% (offline), 5.0% (Mistral 7B), and 3.0% (Llama2-13B). A permutation test (R = 5000), confidence intervals, and power analysis were added. Quality impact statements were refined, with “minimal impact” used instead of “imperceptible.” In this study, limitations and ethical use are discussed, and directions for stronger semantic channels and model-based detectors are outlined.