acceptodds
Under review as a conference paper at ICLR 2027

Towards Statistically Optimal and Semantically Robust Watermarking for Large Language Models

Abstract

The widespread adoption of large language models (LLMs) in real-world applications poses critical challenges for reliably detecting LLM-generated text. Logit-based watermarking is a prominent approach that enables low-overhead statistical detection by biasing token selection through keyed binary masks. However, existing methods often suffer from limited detection signals under constrained perturbations, and remain vulnerable to content-preserving attacks. To address the detectability limitation, we theoretically establish a first-order characterization of watermark detectability in terms of the predictive probability mass induced by the vocabulary partition and derive a closed-form optimality condition that maximizes the expected detection -score under a fixed perturbation strength. Building on this result, we propose SoptiMark, which combines a Cumulative Probability Mask that approximates this optimal allocation with an Embedding Clustering mechanism that stabilizes mask reconstruction across semantically related tokens. This design strengthens the standardized watermark signal while improving resilience to content-preserving modifications. Extensive experiments demonstrate that SoptiMark consistently outperforms prior logit-based watermarking methods by preserving substantially stronger detection signals under diverse content-preserving attacks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Watermarking, Detectability, Robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DtPTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SoptiMark, an improved version of Green/Red watermark (KGW). The method introduces two key components. First, a Cumulative Probability Mask maximizes the first-order watermark detection signal by partitioning the vocabulary such that the green and red lists each contain approximately half of the model's predictive probability mass. Second, Embedding Clustering assigns semantically similar tokens to shared hash keys, improving watermark consistency under token substitutions and paraphrasing. Experiments across multiple LLMs demonstrate that SoptiMark achieves stronger detection performance and greater robustness against text modifications than existing watermarking methods.

Critical strengths

I think the biggest strength of this paper is that it tries to improve watermark from the perspective of vocabulary partitioning and watermark key generation, which I find to be a very interesting and promising direction. In particular, I like the idea of incorporating semantic information into watermark key generation rather than relying solely on token identities. Regardless of the actual performance improvements, I think this is a good starting point for improving the robustness of existing watermarking methods.

Critical weaknesses
  1. Limited practical significance of the theoretical analysis. While the theoretical result suggests that balancing the probability mass between green and red tokens maximizes the detection signal, I am not fully convinced of its practical significance. Beyond motivating a relatively straightforward balancing strategy, I don't see much additional insight or guidance (especially for practice) provided by this analysis.

  2. Difficulty in controlling semantic similarity. I think the biggest challenge is how to properly define and control semantic similarity. Simply clustering token embeddings does not guarantee that tokens within the same cluster have similar meanings. There is also a clear trade-off: using smaller clusters makes the method closer to token-identity-based watermarking, while larger clusters may group semantically different tokens together. I feel this trade-off is difficult to control in practice and is not sufficiently addressed in the paper.

Questions for authors
  1. While the theoretical analysis shows that balancing the probability mass between green and red tokens is optimal, what is the practical significance of this result? (see weakness 1)

  2. How can we properly control semantic similarity through clustering? (see weakness 2)

  3. The authors mention in the Introduction that logit-based watermarking has low overhead. However, since semantic meaning is context-dependent, is offline clustering based on static token embeddings sufficient? If context-aware clustering is needed (since the token semantic should depend on the context), wouldn't that introduce substantial computational overhead and potentially undermine the low-overhead advantage of traditional watermarking methods?

6 comments
AuthorsOpenReview ↗

Response to Reviewer DtPT (1/4)

We appreciate the time and effort you have put into evaluating our work. Your questions prompted us to examine the clustering design more closely and add experiments to clarify its practical benefits and trade-offs. We provide our responses below.

W1/Q1: Practical significance of the analysis.


A practical implication of Theorem 4.1 is quantitative guidance for quality-constrained watermark design. When text-quality requirements limit the acceptable watermark bias, the theorem helps determine how much detection signal can be retained by adjusting the partition instead. Ignoring higher-order terms, retaining at least a fraction rr of the maximum first-order signal requires T−1∑t(PG,t−1/2)2≤(1−r)/4T^{-1}\sum_t(P_{G,t}-1/2)^2\leq(1-r)/4. If every position attains the same mass pp, equality gives p=(1±1−r)/2p=(1\pm\sqrt{1-r})/2. This allows us to choose a partition target according to the desired expected z-score increase, rather than always maximizing the detection signal. The corresponding PPL measurements show the quality cost associated with each target and help identify settings that meet the application’s quality requirements. Our additional experiment reports both quantities across different target masses. For example, the target mass 0.6581 yields an observed mean displacement of 1.4309 with a fixed-context PPL of 2.2174, compared with 1.3123 and 2.3068 for random partitioning. Thus, the analysis provides candidate targets for balancing signal requirements and quality costs, while the quality effects are assessed empirically rather than guaranteed by the theorem.

Target ratio rrTarget cumulative probability massPredicted ΔZ\Delta ZObserved ΔZ\Delta ZFixed-context PPL
10%0.97430.15000.11902.2566
20%0.94720.30050.24082.2368
30%0.91830.45010.36532.2223
40%0.88730.60610.49852.2112
50%0.85360.76880.64082.2037
60%0.81620.93800.79522.1988
70%0.77391.12070.96862.1974
80%0.72361.32411.17122.2023
90%0.65811.56831.43092.2174
100% (SoptiMark)0.50001.94901.93052.3034
Random partitioningNot fixed1.32131.31232.3068
No watermark bias—0.00000.00002.2891

We tested this guidance by varying the target signal fraction and measuring both the resulting detection-score displacement and fixed-context PPL. We used the upper branch, ptarget=(1+1−r)/2p_{\mathrm{target}}=(1+\sqrt{1-r})/2, to construct the cumulative partitions, with token discreteness limiting how closely each target can be reached. This experiment illustrates how the analysis guides the choice of partition targets for different signal requirements and supports empirical assessment of their quality costs, beyond simply recommending a balanced split.

AuthorsOpenReview ↗

Response to Reviewer DtPT (2/4)

W2/Q2 (a): The role of embedding clustering.

We would like to clarify that the primary objective of embedding-based clustering is not to guarantee semantic homogeneity within each cluster, but to improve watermark reconstruction consistency under content-preserving token substitutions. Specifically, embedding similarity serves as a proxy for identifying tokens that may substitute for one another, allowing them to share the same watermark key. Therefore, the relevant design criterion is whether clustering increases green/red membership consistency under plausible substitutions.

To test whether clustering improves this consistency when the conditioning token changes, we conducted a detector-side reconstruction diagnostic under Word-S/50 on 50 texts per model using one fixed watermark key. We evaluated all aligned, unchanged tokens whose immediate predecessor token ID had changed. The variants with and without clustering use the same texts, model probabilities, and positions, retaining the probability-dependent mask construction and changing only the seeding rule. Membership consistency is the percentage of evaluated tokens that retain the same green/red status in the masks reconstructed before and after editing. Clustering increases this consistency by 10.00, 8.16, and 5.85 percentage points on Qwen3, LLaMA, and OPT, respectively. Clustering therefore helps tokens retain their green/red membership when the preceding token is replaced by a synonym.

ModelPositionsMembership consistency w/o clustering (%)Membership consistency w/ clustering (%)Gain (pp)
Qwen3-4B6,62148.7458.74+10.00
LLaMA-3-8B5,95849.8558.01+8.16
OPT-6.7B5,77350.4256.28+5.85
AuthorsOpenReview ↗

Response to Reviewer DtPT (3/4)

W2/Q2 (b): Controlling the clustering granularity trade-off.

To address your concern about controlling clustering granularity in practice, we conducted additional experiments showing why smaller KK is not always preferable: its modest gains in reconstruction consistency can come with increased spoofing risk. Increasing KK can separate related tokens, while decreasing KK can group unrelated tokens together; neither direction resolves both concerns. Our choice of KK is empirical. To provide evidence for assessing this compromise, we added a sensitivity experiment on Qwen3, varying KK from 32 to 2048 while keeping the same 50 Word-S/50 texts, model probabilities, and 6,621 evaluated positions fixed.

Number of clusters KKKey preservation (%)Token membership consistency (%)
3230.9260.31
6430.7160.84
12827.1656.94
25624.7858.65
51225.2758.74
1,02423.6256.09
2,04821.1056.10

The results show that coarser clustering generally improves key preservation, but yields smaller and non-monotonic gains in token membership consistency. For example, reducing KK from 512 to 64 increases key preservation from 25.27% to 30.71%, while membership consistency increases from 58.74% to 60.84%. This suggests that substantially increasing key sharing does not necessarily produce a comparable improvement in reconstruction consistency.

However, coarser clustering may also increase spoofing susceptibility. We simulated a frequency-based spoofing attack on Qwen3-4B, keeping the probability-dependent green-list construction and the watermark bias of 1.25 unchanged across seeding schemes. The simulated attacker knows the model and cluster assignments, but not the watermark key, and receives no detector feedback. At each observation budget, the simulated attacker receives equal numbers of watermarked and unwatermarked reference tokens and uses their conditional frequency ratios to bias the generation of forged texts. ASR is the percentage of forged texts classified as watermarked.

Number of clusters KK / Seeding schemeSpoofing ASR at 2,048 tokens (%)Spoofing ASR at 8,192 tokens (%)
3224.0056.00
6411.0031.00
1286.0016.00
2567.0012.00
5122.335.67
Random partitioning3.003.67

At both evaluated budgets, K=512K=512 has a spoofing ASR close to random partitioning, while all tested coarser configurations have a higher ASR.

Taken together, these results help make the granularity trade-off more concrete. Coarser clustering generally preserves more valid substitution pairs, but its gains in membership consistency are modest and may come with increased spoofing susceptibility. We therefore assess our empirical choice of KK by whether it retains a useful consistency benefit while keeping spoofing ASR close to the baseline across the evaluated budgets.

Reviewer_Fzz4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SoptiMark to improve the detectability and robustness of watermarking. Its first-order analysis shows that the detection signal is maximized when the green and red lists each contain half of the model’s next-token probability mass. Based on this result, it constructs probability-balanced vocabulary partitions. It also groups token embeddings into clusters and assigns a shared partition seed to tokens in the same cluster, aiming to reduce sensitivity to word substitutions. Experiments on three LLMs report improved detection under several editing attacks while maintaining comparable text quality.

Critical strengths
  1. The method addresses both weak initial signals and seed changes caused by text edits, with a separate component targeting each issue.

  2. The experiments cover three model families and several attack types, and include component ablations and a downstream reasoning task.

Critical weaknesses
  1. The motivation is not well connected. The introduction first discusses the trade-off between text quality and detectability, then shifts to robustness against text edits. These are two separate limitations, but their relationship is not clearly explained. The method therefore reads as a combination of two improvements without a strong shared motivation.

  2. Probability-balanced vocabulary partitioning has clear prior work. Balance-Marking (Wang et al., ICLR 2024) already shuffles the vocabulary using a seed and accumulates model probabilities to construct a partition with approximately half of the probability mass. The proposed construction is closely related, with a different cutoff rule.

  3. Mask reconstruction during detection is not sufficiently justified. Algorithm 2 explicitly requires the language model to recompute next-token probabilities. However, the paper does not clearly explain whether the original prompt is available during detection. Text edits can also change these probabilities. Even when embedding clustering preserves the same seed and vocabulary permutation, the probability-based cutoff can change, producing a different mask. Therefore, preserving the seed does not guarantee preservation of the mask.

  4. The reported detection requirements are contradictory. Algorithm 2 requires model inference, and Table 4 reports 17,077 MB of detection memory. Section 6.4 also states that the model is loaded during detection. However, Appendix C.2 claims that detection only loads the tokenizer and the clustering table.

  5. The experiments do not fully establish an improved quality–detection trade-off. Using the same logit bias does not ensure the same text distortion across methods. The main comparisons use fixed or default settings, while Figure 4 varies only SoptiMark. Trade-off curves against strong baselines at comparable PPL or task utility would provide stronger evidence. Robustness comparisons at matched initial watermark strength would also help separate stronger starting signals from better resistance to editing.

Questions for authors
  1. Does detection require access to the original language model and generation prompt? Please clarify how the probabilities are obtained and reconcile Algorithm 2 and Table 4 with Appendix C.2, which states that detection only loads the tokenizer and clustering table.

  2. How does preserving the same cluster key ensure the same mask after editing? Even with an unchanged vocabulary permutation, changes in model probabilities may alter the cutoff. Can you report how often the seed and the resulting mask remain consistent after attacks?

  3. How are detection thresholds calibrated for the reported TPR@1% results? How should the FPRs in Table 6, including 17% for 256 clusters and 3.2% for the default 512 clusters, be interpreted relative to this operating point?

  4. Do the improvements remain when baselines are tuned to comparable text quality? For robustness evaluation, can you also compare methods at matched initial watermark strength to separate stronger starting signals from better resistance to editing?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Fzz4 (1/3)

Thank you for your careful reading and helpful comments. We provide clarifications and additional experiments below, grouping related weaknesses and questions to avoid repetition.


W1: Shared motivation: watermark evidence that remains detectable after editing.

We view the quality-detectability trade-off and editing robustness as connected aspects of the same problem. A stronger initial detection signal provides a larger margin above the detection threshold, allowing the watermark to tolerate more signal degradation from editing. Simply increasing the watermark bias can enlarge this margin, but may compromise text quality. We therefore seek to strengthen the detection signal at a fixed logit-bias magnitude. This motivates our cumulative probability mask, whose first-order signal is characterized in Theorem 4.1. The ablation in Table 3 supports this connection, showing that the cumulative mask alone improves editing robustness over conventional random binary masking, such as KGW.

A complete watermarking scheme includes both injection and detection. The cumulative mask addresses the injection stage, but the resulting signal must also remain recoverable during detection. Editing can change the seed and cause the detector to reconstruct a different vocabulary permutation, weakening the recovered signal. We therefore introduce embedding clustering to reduce this additional loss. Our two components thus improve robustness by strengthening the signal during injection and reducing permutation mismatches between generation and detection. We will revise the introduction to make this connection explicit.


W2: Contribution compared to Balance-Marking.

The distinction concerns the role of probability balancing in the design objective. Balance-Marking introduces cumulative partitioning to mitigate the quality degradation associated with multi-bit message encoding: the selected subset should retain sufficient model probability while supporting discrimination among messages. Our starting point is the statistical efficiency of binary logit watermarking. Specifically, we analyze how the probability mass of a mask determines the detection signal obtained from a fixed perturbation magnitude. Theorem 4.1 makes this dependence explicit: the first-order signal contains a penalty proportional to ∑t(PG,t−0.5)2\sum_t(P_{G,t}-0.5)^2. Probability balance therefore emerges as an optimality condition for this detection objective, with a quantitative characterization of the loss when exact balance is infeasible.

This difference in objective is reflected in the cutoff rule. Balance-Marking selects the first prefix reaching the target mass, whereas our construction selects the prefix minimizing the derived imbalance penalty. For the same distribution and permutation, this maximizes the first-order signal among prefix-based binary masks. Our contribution is thus the detection-signal characterization and the resulting optimization criterion, rather than the use of cumulative probabilities alone. We will make this relationship explicit in the revised discussion.


W3/W4/Q1: Language-model and prompt requirements.

Thank you for pointing out the inconsistency in Appendix C.2. We have corrected the statement in Appendix C.2 to make these requirements consistent throughout the manuscript. Both watermark generation and detection require loading the language model, tokenizer, and clustering table, as specified in Section 6.4 and reflected in the detection memory reported in Table 4.

Our method follows the logit-based watermarking framework, requiring the detector to receive the prompt together with the continuation. Given the prompt PP and generated continuation YY, we concatenate them into X=P∥YX=P\Vert Y and run the language model on the full sequence. At each scored position tt of XX, we use the next-token distribution pθ(⋅∣X<t)p_\theta(\cdot\mid X_{<t}) to reconstruct the probability-based mask.

Regarding your concern that editing can change the input prompt, probabilities, and hence the reconstructed mask, our editing evaluation covers the entire sequence, including both PP and YY. The detector processes the edited sequence X~=P~∥Y~\tilde X=\tilde P\Vert\tilde Y and recomputes pθ(⋅∣X~<t)p_\theta(\cdot\mid\tilde X_{<t}) at each scored position. Thus, our evaluation does not assume that the prompt or the original probability distributions remain unchanged. Embedding clustering preserves the vocabulary permutation for substitutions within the same cluster, but the probability-dependent cutoff may still change. Its role is to reduce permutation mismatches, while the remaining mask changes are included in our empirical evaluation.

AuthorsOpenReview ↗

Response to Reviewer Fzz4 (2/3)

Q2: Seed and mask consistency after editing.

Our clustering mitigates editing-induced mismatches by preserving the seed and vocabulary permutation for substitutions within the same cluster. This prevents such substitutions from changing the vocabulary ordering used during mask reconstruction. Changes in model probabilities can still shift the cumulative cutoff, so we separately measured seed consistency, mask consistency, and whether the scored token retains its green/red membership.

We measured these quantities on three models, using 50 texts and 48 aligned, unchanged-token positions per text for each attack, yielding 2,400 positions per condition. We reconstructed masks from the original and edited prefixes to isolate the effect of context changes on the same scored token. In the table, exact mask consistency requires identical green lists; approximate mask consistency requires their Jaccard similarity to be at least 0.80; token membership consistency checks only whether the scored token retains its original color. All values are proportions.

ModelAttackSeed consistencyExact mask consistencyApprox. mask consistencyToken membership consistency
Qwen3-4BChar-S/300.81880.26420.65290.8350
Char-S/500.67500.19040.50210.7742
Word-S/500.68620.16330.46170.7604
LLaMA-3-8BChar-S/300.78080.23830.64830.8150
Char-S/500.64210.16120.47330.7517
Word-S/500.67670.17330.50380.7612
OPT-6.7BChar-S/300.78790.26170.65830.8279
Char-S/500.66000.18830.50960.7638
Word-S/500.70420.18670.50960.7683

Our measurements show that the seed remains unchanged at 64.21%-81.88% of evaluated positions. The reconstructed green lists remain exactly identical at 16.12%-26.42% of positions, while 46.17%-65.83% retain a green-list Jaccard similarity of at least 0.80. Thus, preserving the cluster key maintains the permutation but does not ensure an identical mask after editing. The scored token nevertheless retains its green/red membership at more than 75% of positions.


Q3: Threshold calibration and the FPRs in Table 6.

Table 1 uses method-specific thresholds calibrated to an empirical FPR of 1%. Each watermarking method has its own detection statistic and threshold. We therefore adjust the threshold separately for each method to compare TPR at the same false-positive operating point, rather than applying a common numerical threshold.

Table 6 reports uncalibrated FPRs, not results at the 1% operating point. We did not recalibrate the threshold for each cluster configuration in that table. The reported 17% FPR for 256 clusters and 3.2% for 512 clusters are therefore the observed false-positive rates under those settings. Their corresponding TPRs should not be directly compared with the calibrated TPR@1% values in Table 1. We have now recalibrated each cluster configuration to target an empirical FPR of 1%, and provide the updated results below.

Number of clustersOriginal FPR (%)Original TPR (%)TPR at empirical 1% FPR (%)
320.474.282.2
640.277.682.8
1289.098.897.8
25617.099.097.0
5123.2100.099.6
10242.897.495.8
20483.696.093.2

We will replace Table 6 with these calibrated results and clarify the threshold-selection procedure in the manuscript.

AuthorsOpenReview ↗

Response to Reviewer Fzz4 (3/3)

W5/Q4 (a): Detection and robustness at comparable text quality.

We considered the quality-detection trade-off when developing the cumulative probability mask in Section 4. Figure 1(c) shows that, across different perturbation strengths, our cumulative partitioning achieves higher z-scores than traditional random partitioning while maintaining comparable text quality. Higher z-scores indicate stronger detection signals and, at a fixed detection threshold, are associated with higher TPR. To complement this analysis, we have added the baseline comparison missing from Figure 4, evaluating KGW (random partitioning) and SoptiMark (cumulative probability partitioning) across multiple perturbation strengths, as reported below.

MethodPerturbationClean TPR@1% (%)Word-S/50 TPR@1% (%)Char-S/50 TPR@1% (%)PPL
KGW179.202.004.206.9178
SoptiMark197.607.0012.606.9444
KGW299.809.0023.607.9585
SoptiMark2100.0023.4037.808.0203
KGW5100.0061.0076.2012.7677
SoptiMark5100.0067.2090.4011.3134
KGW10100.0073.2090.0017.9454
SoptiMark10100.0085.6098.6014.5319

Our results demonstrate these improvements while maintaining comparable text quality. At perturbation 1, KGW and our method have nearly identical PPL, while our method achieves a higher TPR under both Word-S/50 and Char-S/50. The same pattern holds at perturbation 2. At perturbations 5 and 10, our method achieves both lower PPL and higher attacked TPR. Thus, the robustness gains in these comparisons are not obtained by accepting worse text quality, consistent with the trend shown in Figure 1(c).

W5/Q4 (b): Separating initial signal strength from resistance to editing.

Regarding matched initial watermark strength, we consider the stronger starting signal to be part of our robustness contribution. Our cumulative probability mask strengthens the initial signal, while embedding clustering reduces permutation mismatches caused by editing. To examine the contribution of clustering without the signal enhancement introduced by cumulative partitioning, we compare SoptiMark without the cumulative probability mask against the baselines below. All detection results are TPR at 1% FPR, reported as proportions.

MethodCleanWord-S/50Char-S/50Rewrite-DSRewrite-DPPL
KGW0.94400.03840.07260.31220.01637.386
WatME0.95200.04680.06120.34850.02017.214
SWEET0.97610.23400.21660.46290.11687.027
EWD0.98620.24010.27660.50720.12287.170
PF0.98600.06250.04890.18920.04877.432
SIR0.94480.29400.23280.45410.24297.160
k-SemStamp0.97280.28640.24550.42180.21168.320
DiPmark0.95600.01880.03630.04870.01657.083
MorphMarkexp0.96440.24610.10450.45000.12427.186
SoptiMark w/o cumulative mask0.93800.32840.35670.47820.21227.148
SoptiMark0.99600.35490.40810.58260.23277.218

The robustness benefit persists after removing the cumulative probability mask. This variant has a clean TPR of 0.9380, close to KGW’s 0.9440, yet it achieves substantially higher post-edit TPR under all four attacks. It also ranks among the best-performing methods under Word-S/50 and Char-S/50, despite its lower clean TPR.


Thank you again for your constructive feedback. We hope that our responses and revisions have adequately addressed your concerns.

Reviewer_oJT7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is based on the KGW watermark, where green and red lists of tokens are separated using a pseudo random function, and logits of green list tokens are biased with a fixed logit. Compared to KGW, SoptiMark first splits the green and red lists with probability mass because, based on Theorem 4.1, an equal probability split triggers better statistical guarantees. A clustering-based method is applied to select seeds to improve the robustness of the watermark.

Critical strengths

The experimental results are complete.

  1. The evaluation spans 9 baselines, 3 LLMs, and three attacks. The paper also reports downstream performance on GSM8K and computational cost.

  2. The ablation study attributes the contribution to each component. The cumulative probability mask mainly improves the z-score, and the embedding clustering mainly drives robustness.

Critical weaknesses
  1. The basic idea of this paper looks similar to the alpha-reweight method in previous work [1]. Specifically, alpha-reweight applies a permutation to the vocabulary and splits it along the permuted order at cumulative probability (alpha).

  2. Replacing token-level hashing with cluster-level keys shrinks the seed space from ∣V∣|V| to only KK (512 for Qwen/LLaMA, 128 for OPT). Each cluster always induces the same vocabulary permutation. As a result, the green lists can be estimated from fewer watermarked outputs. This substantially lowers the cost of watermark stealing and spoofing attacks, in which an attacker can fully recover the watermark rules.

Questions for authors
  1. Please further clarify the contribution of this study compared to other work, including [1].

  2. How does the cumulative probability mask behave in low-entropy steps where the largest token probability exceeds 0.5? The cutoff always falls immediately before or after the top-1 token.

[1] Wu, Yihan, et al. "Dipmark: A stealthy, efficient and resilient watermark for large language models." (2023).

5 comments
AuthorsOpenReview ↗

Response to Reviewer oJT7 (1/2)

Thank you for your thoughtful comments on the relationship to DiPmark, the security implications of cluster-level key reuse, and the behavior in low-entropy contexts. We address these points below.


W1 / Q1: Contribution compared to DiPmark.

The key distinction is how cumulative probabilities are used. In DiPmark, the probability quantile α\alpha controls the reweighting transformation and is distinct from the red/green separator. In SoptiMark, the cumulative probability directly determines the binary mask to which the logit bias is applied. DiPmark’s alpha-reweight truncates and redistributes probability mass, whereas SoptiMark applies a finite logit perturbation to the selected tokens. Theorem 4.1 provides the rationale for our construction. At a fixed perturbation magnitude, a probability-balanced mask maximizes the first-order watermark detection signal.

Truncation versus binary logit perturbation. DiPmark's alpha-reweight removes probability mass in [0,α][0,\alpha] along the permuted cumulative distribution and rescales the remainder, with fractional treatment of the boundary token. Its DiP-reweight combines two such transformations to achieve distribution preservation in expectation over random permutations. Our cumulative partition instead defines a binary mask for finite logit perturbation:

pt′(i)=pt(i)eϵ1[i∈Gt]1+(eϵ−1)PG,t.p'_t(i)=\frac{p_t(i)e^{\epsilon\mathbf{1}[i\in G_t]}}{1+(e^\epsilon-1)P_{G,t}}.

Thus, SoptiMark preserves positive probability for every token with pt(i)>0p_t(i)>0, including tokens outside the green set. The cumulative cutoff determines which tokens receive the bias, rather than which probability mass is discarded.

A theoretically motivated partition. Theorem 4.1 characterizes the first-order green-count statistic displacement at a fixed logit-bias magnitude. With qt=PG,tq_t=P_{G,t},

ΔZT=ϵT2−2ϵT∑t(qt−12)2+O(ϵ2T).\Delta Z_T=\frac{\epsilon\sqrt T}{2}-\frac{2\epsilon}{\sqrt T}\sum_t(q_t-\tfrac12)^2+O(\epsilon^2\sqrt T).

The leading signal is maximized when each binary mask has a probability mass of 0.5. This yields an explicit imbalance penalty and motivates selecting the whole-token prefix whose probability mass is closest to 0.5. Our contribution is therefore a signal-based justification and construction of probability-balanced binary logit masks, rather than cumulative truncation. The optimality statement holds to first-order within this perturbation family; token discreteness can prevent attaining the ideal balance, as discussed in our response to Q2. We will clarify this scope and connection to DiPmark.

AuthorsOpenReview ↗

Response to Reviewer oJT7 (2/2)

W2: Cluster reuse and stealing/spoofing.

Cluster-level keying reduces permutation diversity, but a reused permutation does not define a fixed green list across contexts. The green list also depends on the next-token probability distribution. This context dependence does not rule out learning approximate token preferences for spoofing, which we evaluate below.

We compared 512-cluster seeding with a token-level control on Qwen3-4B, keeping the probability-dependent cutoff and watermark bias unchanged. We simulated a frequency-based spoofing attack with access to the model and cluster assignments, but not the keys or detector feedback. Using equal budgets of watermarked and clean reference tokens, the attack estimates token-frequency ratios conditioned on the preceding token or cluster and uses them to bias generation. We tested three keys and 300 spoofed texts per key–budget condition. ASR is the percentage of spoofed texts accepted as watermarked, with detection thresholds calibrated on a separate clean set to target 1% FPR.

Watermarked-token budgetKey 1: token / clusterKey 2: token / clusterKey 3: token / clusterMean: tokenMean: clusterΔ (pp)
2,0483.00 / 2.332.67 / 5.333.67 / 1.003.112.89−0.22
8,1923.67 / 5.676.33 / 7.677.33 / 4.005.785.780.00
30,0005.67 / 13.0012.00 / 16.008.33 / 17.008.6715.33+6.67
100,00012.00 / 27.0013.00 / 26.3317.67 / 26.0014.2226.44+12.22

The effect of cluster reuse depends on the number of watermarked tokens observed by the attacker (i.e., the watermarked-token budget). At budgets of 2,048 and 8,192, the average ASRs are comparable; at larger budgets, the attacker benefits from pooling observations by cluster. This evaluation assumes that the attacker has access to both the model and the token-to-cluster mapping, and measures spoofing success. Fully recovering the watermark rules, however, requires resolving mask membership even for rarely sampled tokens. At a fixed watermark bias, distinguishing two candidate masks that differ only on tokens with probability pp can require Ω(1/p)\Omega(1/p) samples at the same prefix, making exact rule recovery highly challenging for low-probability tokens. We will include these results alongside the editing-robustness evaluation to clarify the trade-off under this access setting.


Q2: Low-entropy steps with top-1 probability above 0.5.

When the top-1 token has a probability of r>0.5r>0.5, our cumulative mask selects the cutoff immediately before or after that token, whichever yields a cumulative probability closer to 0.5. Let aa denote the cumulative mass preceding the top-1 token. The two candidate green masses are aa and a+ra+r: the token is assigned to the green set if a<(1−r)/2a<(1-r)/2, and to the red set if a>(1−r)/2a>(1-r)/2, with either choice admissible in a tie. The construction therefore remains well-defined, although exact probability balance is unattainable.

Assignment to the red set does not prohibit sampling the top-1 token. It receives no additive logit bias and retains probability r/[1+(eϵ−1)q]r/[1+(e^\epsilon-1)q], where q≤1−rq\leq1-r is the green-set mass. For fixed finite bias, this probability approaches 11 as r→1r\to1. Thus, in nearly deterministic contexts, the dominant token remains overwhelmingly likely even when assigned to the red set, while the watermarked and original next-token distributions become increasingly similar.

The limited capacity to introduce a detection signal in nearly deterministic contexts also affects conventional random partitioning logit watermarks. It reflects the limited sampling alternatives available in low-entropy contexts. SWEET (Lee et al., 2024), for example, addresses this difficulty by excluding low-entropy positions from watermark embedding and detection. Our cumulative construction seeks the prefix whose probability mass is closest to 0.5 under the keyed permutation, but remains subject to the limited watermarking capacity of nearly deterministic steps.


We appreciate the opportunity to clarify these distinctions and provide additional evidence in response to your concerns.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer oJT7,

We hope this message finds you well. As the discussion period is nearing its end with only two days remaining, we wanted to check whether you have had a chance to review our rebuttal and whether there are any further questions or comments we can address to assist your evaluation. We would be happy to clarify any remaining issues. Thank you again for your time and suggestions for improving our manuscript.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.