acceptodds
Under review as a conference paper at ICLR 2027

FissionGuard: Testing Local Watermark Proportions with FDR Control

Abstract

A watermark detector can flag a text even when only a small fraction of its tokens are watermarked. We study sequential auditing to identify local passages whose watermark proportions exceed a prespecified threshold. Each alarm fixes a candidate passage's endpoint, while its start is chosen from earlier tokens. Reusing the same evidence to locate and test a passage can invalidate inference. We propose FissionGuard, which uses data fission to construct an alarm value and a validation value from each token-level -value. The alarm values guide alarms and localization. The validation values yield passage-level -values through partial conjunction, excluding as many of the strongest signals as the proportion null allows. These -values are used to test the corresponding hypotheses at levels assigned by an online multiple-testing procedure. Under the stated watermark generation model, allowing dependence among token-level -values at watermarked positions, we prove finite-sample false discovery rate (FDR) control among reported passages at every fixed finite horizon. We also derive a finite-sample lower bound on power for testing nonnull candidates. Experiments on synthetic streams and Mistral and Qwen generations across four marking layouts show gains in watermarked-token coverage over sample splitting, with empirical FDR below the target.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
LLM watermark detection, data fission, post-selection inference, partial-conjunction testing, online false discovery rate control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_n8mkOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies how to identify passages whose watermark proportions exceed a chosen threshold. FissionGuard uses data fission to separate the evidence for locating passages from the evidence for testing them. It then uses partial-conjunction tests to account for the watermarked tokens allowed under the null, followed by online multiple testing. The paper proves FDR control at fixed finite horizons under stated assumptions and gives power bounds. Simulations and experiments on Mistral and Qwen show improved token coverage over sample splitting.

Critical strengths
  1. The paper addresses a useful problem. Detecting watermark presence does not establish that a substantial fraction of a passage is watermarked. Testing a chosen proportion threshold gives a more informative claim about mixed-source text.

  2. The method handles selection and testing carefully. Data fission separates passage localization from validation, while partial-conjunction testing accounts for the strong signals allowed under the null. The FDR argument clearly connects these steps under the stated assumptions.

  3. The experiments support the main idea. The paper checks calibration near the proportion-null boundary and shows coverage gains over sample splitting. For example, on Mistral text with random marking, coverage increases from 87.7% to 95.3%, with empirical FDR below the target.

Critical weaknesses
  1. The text experiments need independent validation. Appendix E states that the model-generated texts were used during method development and that the main temperatures were chosen based on watermark evidence strength. This limits how confidently the reported gains can be extended to new texts and generation settings.

  2. The practical requirements need clearer guidance. The experiments use position-dependent seeds, with source identifiers added for internally mixed texts. The paper should clarify what position and request information a detector needs when applying the method across multiple texts.

  3. Sensitivity to the detection settings is not fully studied. The experiments fix the fission allocation and the alarm and localization windows. More guidance is needed on choosing these settings and on how they affect coverage and purity, especially for short passages or weak watermark signals.

Questions for authors
  1. Independent evaluation. Do the coverage gains remain on held-out prompts after all method settings are fixed? In particular, how well does the method perform on internally mixed text at lower generation temperatures?

  2. Requirements for practical use. What position and request information must be available to reproduce the experimental seeds? How should randomness reuse be prevented when the same key is used across multiple requests?

  3. Choice of the detection settings. How should users choose the fission allocation and scanning windows for a new application? Are the coverage gains over sample splitting stable across reasonable choices of these parameters?

4 comments
AuthorsOpenReview ↗

Response to Reviewer n8mk (Part I)

W1. (a) The text experiments need independent validation. Appendix E states that the model-generated texts were used during method development and that the main temperatures were chosen based on watermark evidence strength.

Response. Thank you for your comments. The Table 2 temperatures were fixed before generating the original internally mixed texts. The original corpora were used in development, so we evaluated new held-out texts with fixed detector settings. Our response to your Q1 reports the results.

We apologize for the ambiguity in Appendix E. Earlier detector comparisons informed the Table 2 temperatures. We chose each lower temperature before generating its held-out texts. We will clarify this description in Appendix E.

(b) This limits how confidently the reported gains can be extended to new texts and generation settings.

Response. The coverage gains persisted on new held-out texts at the additional lower temperatures. Our response to your Q1 reports the results, including the lower temperatures for each model.

W2. The practical requirements need clearer guidance. The experiments use position-dependent seeds, with source identifiers added for internally mixed texts. The paper should clarify what position and request information a detector needs when applying the method across multiple texts.

Response. We appreciate your comments. The detector needs the original generation request ID and token positions, even when the monitoring request boundaries change. We will make this explicit in Appendix E. Our response to your Q2 gives the full requirements and verifies that the token-level p-values are reproduced exactly.

W3. Sensitivity to the detection settings is not fully studied. The experiments fix the fission allocation and the alarm and localization windows. More guidance is needed on choosing these settings and on how they affect coverage and purity, especially for short passages or weak watermark signals.

Response. Thank you for your careful reading. We use ρ=0.5, a minimum alarm spacing of 64 tokens and a minimum localization length of 32 tokens. The 64-token spacing lets the alarm rule accumulate evidence after each restart and matches its shortest scanning window. The 32-token minimum allows localization to select a shorter passage within that window. FissionGuard had higher token coverage on both models across the allocation and window choices reported in our response to your Q3.

At ρ=0.5, fission allocates equal information to alarms and validation. Larger ρ gives more information to alarms and localization and less to validation.

For shorter expected passages, the smaller alarm spacing and localization length tested in our response to your Q3 can improve recovery. These choices preserve the FDR guarantee in Theorem 3.3.

The paper’s controlled simulations show that weaker watermark signals reduce recovery with the detector settings fixed, consistent with the dependence on signal strength in the power bounds of Section 3.3. Our response to your Q1 reports lower-temperature text results.

Q1. (a) Independent evaluation. Do the coverage gains remain on held-out prompts after all method settings are fixed?

Response. Thank you for this helpful suggestion. With both detectors fixed before evaluation, FissionGuard had higher token coverage than 50/50 Splitting on held-out prompts for every internally mixed layout and temperature in the table below.

(b) In particular, how well does the method perform on internally mixed text at lower generation temperatures?

Response. The coverage gains also persisted at the lower temperatures shown below. We generated new texts at each temperature using the same held-out prompts and evaluated them with the detector settings from Table 2.

Token coverage on held-out Mistral-7B-Instruct-v0.3 and Qwen3-8B texts at lower temperatures (FissionGuard / Splitting, %)

ModelGeneration temperatureRandomGappedAlternating
Mistral1.292.72 / 75.3790.47 / 78.2994.02 / 85.15
Mistral1.3595.07 / 87.5592.86 / 83.7495.21 / 92.38
Qwen1.591.09 / 84.1089.43 / 80.2791.25 / 89.17
Qwen1.7595.39 / 90.9794.69 / 85.6995.80 / 95.24

Each cell contains 100 texts. FissionGuard’s empirical passage-level FDR ranged from 0.00% to 0.42% across the table.

AuthorsOpenReview ↗

Response to Reviewer n8mk (Part II)

Q2. (a) Requirements for practical use. What position and request information must be available to reproduce the experimental seeds?

Response. Thank you for these questions. Given the watermark key, the detector needs the original generation request ID, the token’s position in that request and the preceding four token IDs. Using this information, we reproduced every token-level p-value exactly after splitting each internally mixed text into multiple monitoring requests.

(b) How should randomness reuse be prevented when the same key is used across multiple requests?

Response. Assign a unique ID to each original generation request and use it when computing the seeds [1]. Different request IDs keep the hash inputs distinct even when token positions and contexts repeat under the same key.

Q3. (a) Choice of the detection settings. How should users choose the fission allocation and scanning windows for a new application?

Response. We appreciate your comments. We recommend starting with ρ=0.5, a minimum alarm spacing of 64 tokens and a minimum localization length of 32 tokens. Our response to your W3 explains these choices, and the comparisons below show how coverage and purity change when they are varied.

(b) Are the coverage gains over sample splitting stable across reasonable choices of these parameters?

Response. FissionGuard had higher token coverage on both models across the allocations and window settings compared below. On the original Table 2 texts, we matched FissionGuard’s ρ to Splitting’s alarm-side token fraction and also reversed the original odd/even assignment at 50/50. Raising ρ from 0.25 to 0.75 increased purity by about four percentage points on both models and reduced coverage by less than one point.

Token coverage and purity by allocation on the original Table 2 texts (%)

MethodFissionGuard ρSplitting alarm-side token fractionMistral (T=1.5) Cov. / PurityQwen (T=2.0) Cov. / Purity
FissionGuard0.25—96.13 / 80.2796.61 / 80.70
Splitting—25%94.51 / 82.7795.21 / 83.17
FissionGuard0.5—95.88 / 82.7496.59 / 82.98
Original Splitting—50%92.39 / 85.5693.68 / 85.67
Swapped parity—50%90.89 / 85.7493.06 / 85.43
FissionGuard0.75—95.52 / 84.3796.23 / 84.46
Splitting—75%0.02 / 100.000.05 / 100.00

Original Splitting uses odd positions for alarms, and swapped parity uses even positions. At 75%, Splitting made only one report on Mistral and three on Qwen, so its 100% purity reflects very few reports.

To assess window sensitivity, we varied the minimum alarm spacing and minimum localization length on the original Table 2 texts, with the other settings unchanged. Each remaining row changes one.

Token coverage by minimum alarm spacing and localization length (FissionGuard / Splitting, %)

Minimum alarm spacing (tokens)Minimum localization length (tokens)Mistral (T=1.5)Qwen (T=2.0)
64 (default)3295.88 / 92.3996.59 / 93.68
321696.93 / 90.3497.48 / 91.64
641697.85 / 93.2598.33 / 94.19
1283296.24 / 91.6296.63 / 93.24
2563294.21 / 78.6495.89 / 78.97

Reducing only the minimum localization length improved coverage with little change in purity. The longest minimum alarm spacing reduced both token coverage and purity. Empirical passage-level FDR remained below the target.

We generated 1,024- and 2,048-token texts at the Table 2 temperatures, scaling the watermarked regions and gaps in proportion to input length. For each model and input length, we generated 100 texts per layout. We compared the Table 2 defaults with the 32/16 setting (alarm spacing/localization length) above.

Token coverage pooled across the four layouts (%, Table 2 defaults → minimum alarm spacing 32 and minimum localization length 16)

Input length (tokens)Mistral (T=1.5) FissionGuardMistral (T=1.5) SplittingQwen (T=2.0) FissionGuardQwen (T=2.0) Splitting
2,04892.73 → 95.1284.83 → 87.6694.04 → 95.7888.30 → 90.08
1,02483.81 → 90.1458.83 → 71.6485.04 → 91.6866.75 → 79.12

The 32/16 setting increased coverage and purity for both methods, with larger coverage gains on the 1,024-token texts. FissionGuard continued to cover more watermarked tokens. Empirical passage-level FDR remained below the target.

Our response to your W3 discusses the paper’s weak-signal results. Our response to your Q1 reports lower-temperature text results.

References

[1] Christ, M., Gunn, S., & Zamir, O. (2024). Undetectable watermarks for language models. In S. Agrawal & A. Roth (Eds.), Proceedings of Thirty Seventh Conference on Learning Theory (Vol. 247, pp. 1125–1139). PMLR.

Thanks for the detailed response. I have maintained my rating .

Thank you for your careful review and taking time to read our response. We appreciate your helpful suggestions.

Reviewer_FVGMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces FissionGuard to test whether adaptively selected passages exceed a specified watermark proportion. I find this topic relevant, and the specific problem remains important in my view despite the growing literature on watermark detection in the last 3 years. The authors use data fission to separate selection from validation, with validation discarding the strongest signals permitted under the proportion null. Under the stated assumptions, the method provides finite-sample FDR control and power bounds for selected candidates. The experiments show coverage gains over sample splitting, with empirical FDR below the target level. I like the statistical motivation and the careful treatment of selection bias and repeated testing. My main concerns are the practical applicability of the independence assumptions, reuse of development corpora, and sensitivity to generation temperature.

Critical strengths

In my own reading, the main strength is the careful formulation of local watermark-proportion testing and the statistical reasoning supporting it. I particularly like how Sections 3.1–3.2 (pp. 4–5) combine data fission, partial-conjunction testing, and online multiple testing to address selection bias, the watermarked tokens permitted under the null, and repeated reporting. I find the argument connecting Proposition 3.2 to the finite-sample FDR guarantee in Theorem 3.3 convincing under the stated assumptions. The power analysis in Section 3.3 (p. 6) and the coverage gains over sample splitting in Tables 1–2 (pp. 8–9) further strengthen the contribution.

Critical weaknesses

One detail that caught my attention is how the method handles request endings. Algorithm 1 (p. 6) discards the remaining tokens at a request boundary without forming a final candidate. This can leave recent watermarked content untested. Recovery may therefore depend on where request boundaries fall. Although the appendix examines request protocols, I would appreciate a direct assessment of the coverage cost of this design choice.

Questions for authors

1.Could the request-replay analysis in Table 7 be extended to mixed-watermark streams? The replay could shift request boundaries while keeping the token-level evidence and fission noise fixed. I would like to understand how much coverage changes when watermarked content falls near, or crosses, a request boundary.

  1. Could a final candidate be tested at each prespecified request end? In my understanding localization would use only alarm values. The test would use previously unused validation values, and the online testing history would be retained. Would this remain compatible with Theorem 3.3? A small comparison with the current discard policy could show whether the additional coverage justifies the extra testing.
4 comments
AuthorsOpenReview ↗

Response to Reviewer FVGM (Part I)

W1. One detail that caught my attention is how the method handles request endings. Algorithm 1 (p. 6) discards the remaining tokens at a request boundary without forming a final candidate. This can leave recent watermarked content untested. Recovery may therefore depend on where request boundaries fall. Although the appendix examines request protocols, I would appreciate a direct assessment of the coverage cost of this design choice.

Response. We appreciate your comments. Within the prespecified region, qualified coverage fell when the region ended at or crossed a request boundary. Testing a candidate from the untested tail recovered most of this loss and preserved the FDR guarantee.

Table 7 assesses false reports on unwatermarked texts. Our response to your Q1 extends the replay to marked texts to measure the coverage cost, and our response to your Q2 compares discarding the tail with testing it. We will revise the algorithm in the main text to include the request-end test.

Q1. Could the request-replay analysis in Table 7 be extended to mixed-watermark streams? The replay could shift request boundaries while keeping the token-level evidence and fission noise fixed. I would like to understand how much coverage changes when watermarked content falls near, or crosses, a request boundary.

Response. Thank you for this helpful suggestion. Within the prespecified region, qualified coverage was lowest when the region ended at a request boundary, about 29 percentage points below the centered case on each model. Placing a request boundary inside the region also reduced coverage on all three internally mixed layouts.

We extended the Table 7 replay to the 800 marked Table 2 texts. We moved only their monitoring request boundaries, keeping the watermarked positions, token p-values and fission noise fixed. We prespecified one watermarked region in each text and moved the request boundaries around it. The table compares centering the region in a request with placing the boundary just beyond its end, at its end, or at its midpoint. Coverage stayed close to the centered result when the boundary fell 16 tokens beyond the region.

Candidate coverage and qualified coverage (Cov.⁺) within the prespecified region (%)

Position of the regionMistral 1.5 Candidate coverageMistral 1.5 Cov.⁺Qwen 2.0 Candidate coverageQwen 2.0 Cov.⁺
Centered in request99.7095.4299.8096.21
Ends 16 tokens before boundary98.9594.7099.4195.92
Ends at boundary71.2866.6571.2467.37
Crosses boundary at midpoint78.5174.8578.4175.14

Both columns use the watermarked tokens in the prespecified region as the denominator. Candidate coverage counts those in any candidate passage. Qualified coverage (Cov.⁺) counts those in reported passages whose true watermark proportion exceeds η.

AuthorsOpenReview ↗

Response to Reviewer FVGM (Part II)

Q2. (a) Could a final candidate be tested at each prespecified request end? In my understanding localization would use only alarm values. The test would use previously unused validation values, and the online testing history would be retained. Would this remain compatible with Theorem 3.3?

Response. We appreciate this helpful suggestion. Yes, the request-end test preserves the FDR guarantee in Theorem 3.3.

At a prespecified request end, localization uses only alarm values, and the test uses previously unused validation values. Candidate passages remain disjoint. We keep the same online testing history across requests and assign each testing level using only completed tests.

(b) A small comparison with the current discard policy could show whether the additional coverage justifies the extra testing.

Response. The extra tests recovered most of the lost qualified coverage when a marked region ended at or crossed a request boundary. When the region was centered, qualified coverage changed little. We kept the usual alarms and the same request boundaries as in our response to your Q1. If the last token of a request does not trigger an alarm and at least 32 tokens have been observed since the last restart, we use the usual localization rule to choose the start of one candidate ending at that boundary. When the region ended at the boundary, this added about eight tests per text across all its requests.

Qualified coverage (Cov.⁺) when the request tail is discarded or used for a candidate (%)

Region or full textMistral T=1.5 Discard → end testQwen T=2.0 Discard → end test
Centered in request95.42 → 95.3396.21 → 96.08
Ends 16 tokens before boundary94.70 → 95.3395.92 → 96.16
Ends at boundary66.65 → 94.9067.37 → 95.63
Crosses boundary at midpoint74.85 → 95.8675.14 → 96.42
Full text (region ends at boundary)86.61 → 95.7286.92 → 96.21
Full text (one request)95.82 → 95.8296.52 → 96.52

When each text was monitored as one request, the added tests produced one false report and covered no additional watermarked tokens.

Thank you for the detailed responses. This addresses my concerns. I maintain my score.

Thank you for following up. We’re glad our response addressed your concerns. Your suggestion to test the remaining passage at the end of a request led us to add this option in Section 3.2 and evaluate it in Table 8. We appreciate the suggestion.

Reviewer_e2CCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies online reporting of adaptively selected passages whose watermark proportions exceed a prespecified threshold. FissionGuard transforms each token-level p-value into separate alarm and validation values. Alarm values select passage boundaries, while reserved validation values support a partial-conjunction test that allows a specified number of watermarked positions under the null. An online multiple-testing procedure controls passage-level FDR at fixed finite horizons under the stated independence assumptions. The paper also develops power bounds and evaluates synthetic streams and model-generated text.

Critical strengths
  1. The inferential target is well motivated and more specific than watermark presence. The procedure explicitly addresses both adaptive passage selection and contamination permitted under the proportion null.

  2. The validity argument has a clear structure: conditional calibration after fission, disjoint validation sets, partial-conjunction testing, and online multiplicity control. The experiments also distinguish candidate-level power from token coverage and report purity, avoiding conflation of these objectives.

Critical weaknesses
  1. The applicability of the guarantee depends critically on Assumption 3.1, which requires unused watermark randomness to be jointly independent of the complete text and marked randomness. Appendix E discusses practical hash-based generation and possible seed reuse, but these implementation details do not automatically establish the idealized assumption. The extent to which violations affect calibration remains an important practical uncertainty.

  2. The model-generated corpora were used during method development, and the temperatures for internally mixed text were selected partly based on observed evidence strength. These disclosures are helpful, but they limit the strength of the generalization evidence. The restart baselines also do not target the same proportion guarantee, making Splitting the principal matched comparator.

Questions for authors
  1. Could you provide a controlled calibration stress test that varies key/context reuse or dependence strength, while keeping marginal null p-values uniform? This would clarify sensitivity to the joint independence assumption.

  2. Do the coverage gains persist on held-out prompts and keys at independently chosen temperatures? It would also be useful to separate gains attributable to fission from those attainable by varying the allocation and tuning of a valid splitting baseline.

4 comments
AuthorsOpenReview ↗

Response to Reviewer e2CC (Part I)

W1. (a) The applicability of the guarantee depends critically on Assumption 3.1, which requires unused watermark randomness to be jointly independent of the complete text and marked randomness.

Response. Thank you for your careful reading. Assumption 3.1 gives the conditional distribution of validation values in Proposition 3.2, which supports the partial-conjunction test and the FDR guarantee. Li et al. [1] assume independent pseudorandom variables across positions, and Blanchet et al. [2] impose a related conditional-independence assumption when watermark dependence is absent.

In the sufficient model of Appendix A.1, the pseudorandom variables ζₜ are mutually independent across positions and requests and independent of all non-key generation randomness. At unwatermarked positions, ζₜ is not used by current or later generation and is not reused at other positions. Generation does not use the detector’s statistics or decisions. Given the non-key generation randomness and the variables at watermarked positions (t ∈ ℳ), the complete text is determined token by token. The unused ζₜ at unwatermarked positions (t ∈ 𝒩) remain mutually independent with the distributions specified by the watermarking scheme, so they are jointly independent of the complete text and the variables at watermarked positions. Assumption 3.1 itself allows dependence in the text and among the pseudorandom variables at watermarked positions.

(b) Appendix E discusses practical hash-based generation and possible seed reuse, but these implementation details do not automatically establish the idealized assumption. The extent to which violations affect calibration remains an important practical uncertainty.

Response. Appendix E describes the implementation, while Appendix A.1 establishes joint independence under its generation model. We compute each seed from the watermark key, the original generation request ID, the token’s position in that request and the preceding four token IDs. Appendix E omits the original stream ID when describing Contiguous generation and consequently suggests seed reuse across sources. We will correct this description. Including the generation ID and token position keeps the hash inputs distinct when contexts repeat under the same key [3]. We sample unwatermarked tokens using separate randomness.

Our response to your Q1 shows that strong block dependence can inflate empirical FDR even with uniform marginal null p-values. Empirical FDR remained below the target under the tested context-reuse rules, including the replays on internally mixed texts.

W2. (a) The model-generated corpora were used during method development, and the temperatures for internally mixed text were selected partly based on observed evidence strength. These disclosures are helpful, but they limit the strength of the generalization evidence.

Response. Thank you for your comments. The Table 2 temperatures were fixed before generating the original internally mixed texts. The original corpora were used in development. On new prompts and keys, the coverage gains persisted with both detectors fixed before evaluation.

We apologize for the ambiguity in Appendix E. Earlier detector comparisons informed the Table 2 temperatures. We will clarify this description in Appendix E. Each temperature was chosen before generating its held-out texts. Our response to your Q2 reports the results.

(b) The restart baselines also do not target the same proportion guarantee, making Splitting the principal matched comparator.

Response. Splitting tests the same passage-level proportion null as FissionGuard. The restart baselines test for watermark presence. Neither these restart baselines nor the prior watermark detectors discussed in the paper provide passage-level FDR control for adaptively selected passages under this proportion null. We therefore use Splitting as the matched comparator in our response to your Q2, with different assignments and allocations.

References

[1] Li, X., Ruan, F., Wang, H., Long, Q., & Su, W. J. (2025). A statistical framework of watermarks for large language models: Pivot, detection efficiency and optimal rules. The Annals of Statistics, 53(1), 322–351.

[2] Blanchet, J. H., Cai, T. T., Li, X., Liu, H., Long, Q., & Su, W. J. (2026). Optimal watermark localization in mixed-source large language model texts [Preprint]. arXiv.

[3] Christ, M., Gunn, S., & Zamir, O. (2024). Undetectable watermarks for language models. In S. Agrawal & A. Roth (Eds.), Proceedings of Thirty Seventh Conference on Learning Theory (Vol. 247, pp. 1125–1139). PMLR.

AuthorsOpenReview ↗

Response to Reviewer e2CC (Part II)

Q1. Could you provide a controlled calibration stress test that varies key/context reuse or dependence strength, while keeping marginal null p-values uniform? This would clarify sensitivity to the joint independence assumption.

Response. Thank you for this helpful suggestion. Strong block dependence inflated empirical FDR despite uniform marginal null p-values. The tested seed-reuse rules on model-generated texts left empirical FDR below the target.

We first simulated entirely unwatermarked streams. Using a Gaussian copula, we varied block length and within-block correlation while keeping each null p-value marginally uniform.

Empirical FDR under block dependence in entirely unwatermarked streams

Within-block Gaussian correlation4-token blocks16-token blocks64-token blocks
00.00%0.00%0.00%
0.10.00%0.00%1.15%
0.250.00%1.75%20.90%
0.50.10%13.85%71.80%

We repeated the simulation with 16 of every 64 tokens watermarked (η=0.25). At a within-block correlation of 0.25, empirical FDR was 20.72% when only unwatermarked positions were dependent within each block and 25.11% when watermarked positions were also dependent.

To test reuse on model-generated text, we recomputed the token-level p-values for the same 60 unwatermarked texts with the same 128 keys under different hash rules. We ignored token positions when computing seeds and used only the preceding token as context. Under this rule, 29.34% of positions reused a hash input. Of the null p-values, 5.00% fell at or below 0.05, and empirical passage-level FDR was 0.625%.

We then replayed FissionGuard on the 600 internally mixed Table 2 texts under two seed-reuse rules. We defined repeated contexts using either the preceding four tokens or the preceding token. Within each text, we scored each unwatermarked token using the seed from the first unwatermarked position with the same context. The token-level p-values at watermarked positions and the fission noise stayed fixed. Token coverage changed by less than 0.1 percentage point on each model.

Reusing seeds at unwatermarked positions with repeated contexts (Mistral-7B-Instruct-v0.3 at T=1.5, Qwen3-8B at T=2.0, metrics in %)

Seed rule at unwatermarked positionsMistral FDRMistral Cov.Qwen FDRQwen Cov.
Original position + context0.47194.770.38095.42
Reuse on repeated four-token context0.47194.770.38095.42
Reuse on repeated one-token context0.41494.680.30695.41
AuthorsOpenReview ↗

Response to Reviewer e2CC (Part III)

Q2. (a) Do the coverage gains persist on held-out prompts and keys at independently chosen temperatures?

Response. Thank you for these questions. FissionGuard had higher token coverage than 50/50 Splitting on new held-out prompts and keys for all three internally mixed layouts at each additional lower temperature. We chose these temperatures before generating their texts and fixed both detectors before evaluation. Earlier experiments informed some of these temperature choices, as explained in our response to your W2 (a).

The additional lower temperatures were 1.2 and 1.35 for Mistral, and 1.5 and 1.75 for Qwen.

Token coverage (Cov.) on held-out texts at lower temperatures (FissionGuard / Splitting, %)

ModelGeneration temperatureRandomGappedAlternating
Mistral1.292.72 / 75.3790.47 / 78.2994.02 / 85.15
Mistral1.3595.07 / 87.5592.86 / 83.7495.21 / 92.38
Qwen1.591.09 / 84.1089.43 / 80.2791.25 / 89.17
Qwen1.7595.39 / 90.9794.69 / 85.6995.80 / 95.24

Each cell contains 100 held-out texts of 4,096 tokens.

At the Table 2 temperatures, we also compared three 50/50 Splitting rules on the same held-out texts. Original Splitting uses odd positions for alarms and even positions for validation. Swapped parity reverses the assignment, and Random 50/50 assigns each token to alarms independently with probability 0.5.

Token coverage (Cov.) on held-out 4,096-token texts at the Table 2 generation temperatures (%)

Model (temperature)LayoutFissionGuardOriginal SplittingSwapped parityRandom 50/50Gain over original
Mistral (T=1.5)Contiguous97.5196.5996.2796.110.93
Mistral (T=1.5)Random95.1088.1485.9681.796.96
Mistral (T=1.5)Gapped92.9184.9082.8681.058.02
Mistral (T=1.5)Alternating95.4794.2290.3083.071.25
Qwen (T=2.0)Contiguous97.8496.8796.6296.270.97
Qwen (T=2.0)Random95.3990.9190.0585.824.48
Qwen (T=2.0)Gapped94.7486.3185.3582.968.43
Qwen (T=2.0)Alternating95.6795.2793.8286.940.40

For each model, we evaluated 100 texts per layout.

(b) It would also be useful to separate gains attributable to fission from those attainable by varying the allocation and tuning of a valid splitting baseline.

Response. FissionGuard had higher token coverage at each matched allocation. We matched FissionGuard’s ρ to Splitting’s alarm-side token fraction, keeping the texts, windows and alarm threshold fixed.

Token coverage on held-out texts, pooled across the four layouts at η=0.25 (FissionGuard / Splitting, %), with paired gains in percentage points

FissionGuard ρSplitting alarm-side token fractionMistral (T=1.5) Cov.GainQwen (T=2.0) Cov.Gain
0.2525%96.33 / 94.511.8296.59 / 95.371.22
0.550%95.98 / 92.783.2096.54 / 93.812.73
0.7575%95.61 / 0.0195.5996.31 / 0.1296.19

Among these three allocations, Splitting had the highest pooled token coverage when 25% of its tokens were used for alarms. In a separate comparison, we tuned both methods on the same development texts and evaluated the selected settings on held-out texts. FissionGuard’s pooled coverage gain was 1.32 percentage points for Mistral and 1.18 for Qwen.

For η>0, Splitting’s trimming budget is based on the full passage length, although it validates only a subset of the passage. All watermarked tokens allowed under the passage null could fall in that validation subset. FissionGuard keeps validation evidence at every position. When 75% of the tokens are used for alarms and η=0.25, the trimming budget removes all validation values for most Splitting candidates. We also varied η on the original Table 2 texts with 50/50 Splitting. On Gapped texts, FissionGuard’s coverage gain rose from less than one percentage point at η=0 to over eight points on each model at η=0.25. At η=0.5, 50/50 Splitting reported no passages in any layout on either model.

AuthorsOpenReview ↗

Following-up to Reviewer e2CC

Thank you again for your helpful suggestions. In the revised paper, we added the stress tests you suggested to examine how dependence and seed reuse affect empirical passage-level FDR in Appendix E.5.

Tables 9 and 10 report token coverage across temperatures on texts generated from held-out prompts with new watermark keys. We set each temperature before generating the corresponding texts. Table 11 compares three fission allocations, each matched to the fraction of tokens Splitting uses for alarms. In a separate comparison in Appendix E.4, we tuned both methods on the same development texts and evaluated token coverage on held-out texts.

We appreciate your time and would be glad to answer any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading and helpful suggestions. We have marked the revisions in blue.

  • Watermark randomness and seed reuse (Appendices A.1, E.1 and E.5). Appendix A.1 gives a sufficient model for Assumption 3.1. Each position has independent watermark randomness, and text generation uses it only at watermarked positions. In our implementation, the original generation request ID and token position keep hash inputs distinct when a context repeats. In simulations with uniform marginal null p-values, strong block dependence raised empirical passage-level FDR above the 10% target. In replays of 600 internally mixed texts, empirical FDR stayed below 0.5% under both seed-reuse rules tested at unwatermarked positions.

  • Token coverage on held-out prompts (Appendices E.3 and E.4). We evaluated both detectors on texts generated from held-out prompts with new watermark keys. At the Table 2 temperatures, FissionGuard covered more watermarked tokens than 50/50 Splitting on all four layouts for both models. With the same detector settings, the gain persisted on all three internally mixed layouts at every temperature shown in Tables 9 and 10. On Gapped texts, it was at least eight percentage points.

  • Fission allocation and passage length (Tables 11–13). We matched the fission allocation to the fraction of tokens Splitting used for alarms. At η = 0.25, FissionGuard had higher pooled token coverage on both models at all three allocations. With 75% of tokens used for alarms, trimming left most Splitting candidates without validation values. FissionGuard constructs a validation value at every position. For both methods, shorter alarm and localization settings gave larger coverage gains on 1,024-token texts than on 2,048-token texts.

  • Testing at request ends (Section 3.2 and Table 8). Following Reviewer FVGM’s suggestion, we added an optional test at prespecified request ends. We localize using only alarm values, test with previously unused validation values, and keep the same online testing history across requests. The passage-level FDR guarantee in Theorem 3.3 still applies. When a watermarked region ended at the boundary, its qualified coverage rose from about 67% to 95% on both models, with about eight additional tests per text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.