RamoMark: Multi-Bit Watermarking via Randomness-Modulated Speculative Sampling

Speculative sampling accelerates LLM inference through a draft-then-verify paradigm, where proposed tokens are accepted or rejected by comparing the probability ratio of target and draft models against a random threshold. In this paper, we identify this threshold as an intrinsic and controllable source of randomness, and show that multi-bit watermarks can be embedded into the generated text by modulating it-hence unifying efficient inference and output provenance within a single sampling framework. Specifically, we introduce a control variable that modulates the threshold, and construct a communication channel by treating this variable as the input and the generated token as the output. This construction naturally casts multi-bit watermarking as a channel coding problem. Drawing on coding theory, we propose randomness-modulated speculative sampling with watermark (RamoMark), a keyless and robust watermarking algorithm that provably preserves the output distribution. As the codebook is constructed offline, RamoMark scales naturally to long-bitstream watermarking. Empirical evaluations confirm reliable watermark detection with negligible degradation in generation quality and superior performance over baseline methods.