Reducing False Positives in Watermark Detection through Low-Entropy Token Filtering

This thesis addresses the problem of false positives in watermark detection for large language models (LLMs), with a particular focus on open-domain news-style text. As LLMs increasingly generate content that mimics human writing, the risk of mistakenly classifying human-written text as watermarked poses serious ethical and practical concerns. To mitigate this issue, this study investigates two strategies that extend the KGW framework: named entity-based weighting and entropy-based filtering. The proposed methods aim to reduce the influence of highly predictable tokens, which can distort detection signals. While named entity-based weighting leverages the linguistic role of proper nouns and factual expressions, entropy-based filtering excludes low-entropy tokens under the assumption that they carry less discriminative value. Among the approaches explored, a Two-Pass entropy filtering strategy—applied only to texts initially flagged as watermarked—proved particularly e!ective. Specifically, with an entropy threshold of 1.0, this method reduced false positive rates by approximately 2.6 times compared to the baseline, while maintaining low false negative rates. In contrast, the NER-based method demonstrated limited but theoretically interpretable benefits. Empirical analysis of token entropy computed by GPT-2, LLaMA 2-7B, and LLaMA 3.1-8B on human-written texts revealed notable di!erences in entropy distributions, underscoring the need for model-aware detection parameters. Additional analysis confirmed that excluding repeated bigrams improves detection stability and is consistent with prior heuristics. By integrating linguistic and statistical insights into the detection pipeline, this thesis contributes a practical and modular enhancement to existing watermarking frameworks. The findings provide both empirical support and methodological foundations for future research on context-sensitive, low-error watermark detection in machine-generated content.