ChiselMark: Efficient and Detectable Watermarking for Large Language Models under Speculative Sampling

Abstract Generation-time watermarking has become a central safeguard for tracing the outputs of hosted large language models (LLMs) and protecting providers' intellectual property against unauthorized use, yet it must coexist with the increasingly optimized systems through which these models are deployed in production. One such system is speculative sampling, a widely used paradigm for accelerating LLM generation that is now embedded in most production LLM-serving stacks. Here, we study KGW-style logit-reweighting watermarks under speculative sampling and show that straightforward integration systematically erodes the acceleration that speculative sampling is meant to deliver, even when the intended watermarked output distribution is preserved at the token level. We trace this erosion to a latent assumption that current KGW-style methods inherit from standard autoregressive decoding, namely that the distribution being reweighted is also the one whose statistics determine the cost of a write. Under speculative sampling this assumption breaks, because the cost of a write is paid by the proposal side, by the draft distribution at the token level and by the already-constructed candidate tree at the feature level, rather than by the reweighted target. Guided by this understanding, we introduce ChiselMark, a sparse, cost-aware, and architecture-aware watermarking framework that decomposes watermark strength into three distinct controls: where the watermark may be written, whether a particular write is predicted to be locally affordable, and at which stage of the decoding architecture the write enters, while remaining detectable from the generated text, a secret key, and public detector parameters. Across token-level speculative sampling, Medusa, and three generations of EAGLE on multiple model scales and tasks, ChiselMark recovers much of the speculative efficiency lost to dense watermarking while keeping detectability competitive with existing watermarks, and retains detectability under text edits. These findings indicate that practical mechanisms for LLM traceability and controlled deployment should be co-designed with the inference architectures on which they operate, rather than treated as orthogonal interventions at the next-token distribution.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-11525044/v1
Primary Topic
Advanced Steganography and Watermarking Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

ChiselMark: Efficient and Detectable Watermarking for Large Language Models under Speculative Sampling

Yiming Li, Yongcheng Jing, Shizhan Cai, Dacheng Tao et al.
2 links
Research Square
Advanced Steganography and Watermarking Techniques
preprint

ChiselMark: Efficient and Detectable Watermarking for Large Language Models under Speculative Sampling

Yiming Li, Yongcheng Jing, Shizhan Cai, Dacheng Tao, Siyuan Liang
preprint en

Abstract

Abstract Generation-time watermarking has become a central safeguard for tracing the outputs of hosted large language models (LLMs) and protecting providers' intellectual property against unauthorized use, yet it must coexist with the increasingly optimized systems through which these models are deployed in production. One such system is speculative sampling, a widely used paradigm for accelerating LLM generation that is now embedded in most production LLM-serving stacks. Here, we study KGW-style logit-reweighting watermarks under speculative sampling and show that straightforward integration systematically erodes the acceleration that speculative sampling is meant to deliver, even when the intended watermarked output distribution is preserved at the token level. We trace this erosion to a latent assumption that current KGW-style methods inherit from standard autoregressive decoding, namely that the distribution being reweighted is also the one whose statistics determine the cost of a write. Under speculative sampling this assumption breaks, because the cost of a write is paid by the proposal side, by the draft distribution at the token level and by the already-constructed candidate tree at the feature level, rather than by the reweighted target. Guided by this understanding, we introduce ChiselMark, a sparse, cost-aware, and architecture-aware watermarking framework that decomposes watermark strength into three distinct controls: where the watermark may be written, whether a particular write is predicted to be locally affordable, and at which stage of the decoding architecture the write enters, while remaining detectable from the generated text, a secret key, and public detector parameters. Across token-level speculative sampling, Medusa, and three generations of EAGLE on multiple model scales and tasks, ChiselMark recovers much of the speculative efficiency lost to dense watermarking while keeping detectability competitive with existing watermarks, and retains detectability under text edits. These findings indicate that practical mechanisms for LLM traceability and controlled deployment should be co-designed with the inference architectures on which they operate, rather than treated as orthogonal interventions at the next-token distribution.

Research Square
Nanyang Technological University (SG)
Advanced Steganography and Watermarking Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon