acceptodds
Under review as a conference paper at ICLR 2027

Hide in Plain Sight: Towards Scalable and Text-only LLM Copyright Auditing via Watermarking Token-wise Attribution

Abstract

Safeguarding the intellectual property of Large Language Models (LLMs) has become a critical priority. While black-box watermarking offers a proactive solution, existing paradigms face significant limitations: output-space methods inevitably compromise model utility and functional integrity, whereas current alternative-space techniques rely on fine-grained logits and computationally expensive global regression. To address these challenges, we propose CAST, a scalable and text-only auditing framework via watermarking token-wise attribution. Specifically, CAST decouples the embedding process into independent, token-wise attribution and anchors the watermark signal to the importance scores of input tokens on a vocabulary subset . This framework fundamentally resolves the memory bottleneck by achieving a length-independent space complexity and enables reliable text-only verification, where signals are recovered via frequency-based statistical estimation. Extensive experiments on various modern LLMs demonstrate that CAST achieves near 100% watermark success rates in text-only settings while preserving the original model's utility and scalability, significantly outperforming state-of-the-art LLM watermarking methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Copyright Auditing, Model Watermarking, Feature Attribution, AI Copyright Protection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Wyc5OpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CAST is a model-ownership watermarking framework designed for text-only verification. It fine-tunes an LLM so that replacing individual tokens in a secret trigger produces signed changes in the probability that the first generated token belongs to a secret vocabulary subset. These signs encode a multi-bit watermark. Verification estimates the relevant probabilities through repeated text-only queries, while sequential token-wise optimization reduces the number of simultaneously retained computation graphs during embedding. Experiments across several LLMs evaluate bit recovery, utility, distinctiveness, and robustness to model modifications. The approach is practically motivated, but the complete-recovery analysis and the claimed structural memory advantage over EaaW need stronger justification.

Critical strengths
  1. A concrete text-only verification mechanism.

    The vocabulary-subset event can be estimated directly from sampled outputs, without requiring logits from the suspicious API. This is a useful adaptation of attribution-based watermarking to a realistic access restriction, and the method does not require unusually long target responses or a separately trained detector.

  2. Broad empirical coverage and a useful memory–computation trade-off.

    The evaluation covers multiple model families, several watermark lengths, and modifications including fine-tuning, quantization, and merging. The supplementary generative-task evaluation is also useful. Sequential backpropagation provides a genuine reduction in simultaneously retained model activations relative to the tested EaaW implementation, which may make watermark embedding more accessible under limited GPU memory.

Critical weaknesses
  1. The theoretical recovery guarantee and empirical ownership reliability are insufficiently established.

    The complete-recovery proof assumes independence that the extraction algorithm does not provide. Algorithm 2 estimates the original-trigger probability once and reuses it for all bit estimates. Writing these estimates as ϕ^i=p^−q^i\hat{\phi}_i = \hat{p} - \hat{q}_i, independent masked-prompt sampling still gives Cov⁡(ϕ^i,ϕ^j)=Var⁡(p^)\operatorname{Cov}(\hat{\phi}_i,\hat{\phi}_j) = \operatorname{Var}(\hat{p}) for distinct ii and jj. Appendix A nevertheless derives the complete-watermark recovery expression by assuming independent bit extraction. The theorem is therefore not established for the stated algorithm, although this does not imply that the method fails or that the numerical expression is necessarily incorrect. A dependence-aware analysis, a valid union bound, or independently re-estimated baselines would be needed, with the latter increasing the query budget. More broadly, WSR measures bit-matching accuracy rather than ownership-level reliability. The paper does not clearly establish ownership-level TPR/FPR or complete-message recovery across independently sampled triggers, keys, watermarks, and training runs. While the binomial null calculation is valid when the target watermark is independently uniform and independent of null extraction, the corresponding assumptions and uncertainty estimates for the actual ownership decision procedure require clarification.

  2. The practical advantages and distinction from prior attribution-based watermarking require stronger controlled comparisons.

    The memory comparison with EaaW overstates an implementation-dependent limitation. A global regression objective does not inherently require retaining all LLM computation graphs simultaneously: for fixed masks, EaaW's attribution can be expressed as a=Ava = Av, allowing the loss derivatives with respect to vv to be computed before recomputing individual outputs and accumulating model gradients. Although this introduces computation and replay constraints, Table 13 establishes an advantage over the evaluated implementation rather than an unavoidable asymptotic limitation. A recomputation/checkpointing baseline is needed. Moreover, EaaW already investigates label-only verification in Appendix D, albeit for image classification rather than text-only LLM APIs. CAST should therefore distinguish its vocabulary-subset events, token-wise differences, and repeated text sampling from prior attribution-based verification, ideally through a controlled baseline using the same observation mechanism. Practical verification costs also remain substantial: at T=100T=100, a 128-bit watermark requires 12,900 API requests, and single-token responses do not eliminate prompt-processing costs. End-to-end latency and performance under restricted query budgets are not established. Finally, the prefix-attack evaluation does not distinguish model-generated prefixes from prefixes prepended at the API output layer. The latter could disrupt first-token verification without substantially affecting answer utility, so the reported MMLU degradation does not establish an inherent watermark-removal–utility trade-off.

Questions for authors
  1. Theoretical recovery guarantee and empirical calibration.

    How should Theorem 1 be modified to account for the shared original-trigger estimate in Algorithm 2? Under the corrected analysis, what recovery guarantee or empirical calibration is supported at the default sampling budget?

  2. Memory efficiency and attribution-based baselines.

    Does CAST retain a substantial memory–time advantage over EaaW with recomputation or checkpointing? Can the effect of token-wise attribution be separated from that of the vocabulary-subset observation mechanism?

  3. Ownership-level verification reliability.

    How many independent trigger/key/watermark configurations and training seeds underlie the reported WSRs? What are the ownership-level TPR/FPR and complete-watermark recovery rates, with uncertainty estimates, at a prespecified threshold?

  4. Query efficiency and robustness to output-layer manipulation.

    Under a fixed query budget, what verification performance is achieved? For prefix manipulation, does the utility degradation remain when the prefix is added only at the output layer and downstream answers are extracted appropriately?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Wyc5 (1/3)

Dear Reviewer Wyc5,

Thank you for your careful analysis and constructive suggestions! We appreciate your recognition of CAST's concrete text-only verification mechanism, broad empirical evaluation, and practical memory–computation trade-off. We address your concerns below.


W1-1&Q1 Recovery Analysis and Calibration

Thank you for pointing out the dependence between bit estimates!

1. Bit recovery is independent once the shared original-prompt estimate is fixed. Given this estimate, independently sampled masked-prompt responses yield independent bit decisions. We can therefore multiply their conditional recovery probabilities and then average over the shared estimate, without changing CAST or adding queries.

2. We can also bound recovery without assuming bit independence. Keeping Theorem 1's Gaussian approximation and single-bit analysis, a union bound gives

Pr⁡(w^=w)≥max⁡(0,1−k[1−Φ(ϵ2T1−ϵ2)]).\Pr(\hat{w} = w) \geq \max\left(0, 1 - k\left[1 - \Phi\left(\epsilon \sqrt{\frac{2T}{1-\epsilon^2}}\right)\right]\right).

This replaces the product of marginal success probabilities and allows dependent bit-recovery events. We revised Theorem 1 in Section 3.5 and its proof in Appendix A.

3. Removing the independence assumption gives a similar watermark-recovery lower bound. With the same k=128k=128, T=100T=100, and ϵ=0.25\epsilon=0.25:

AnalysisComplete-recovery lower bound (%)
Original product, assuming independent bit recovery98.345
Union bound, without assuming independent bit recovery98.331

The two bounds differ by only 0.014 percentage points, retaining a similar recovery bound without the independence assumption.


W1-2&Q3 Ownership-level Reliability

Thank you for raising this question about verification reliability!

1. CAST maintains stable repeated extraction across different configurations. We vary the trigger, vocabulary key, watermark, and training seed on Qwen3-4B-Instruct-2507. We evaluated eight successfully watermarked models with twenty extractions per model (Appendix D.8, Table 18):

MetricResult (%)
Text-only WSR99.39±0.7699.39 \pm 0.76
Ownership acceptance100.00±0.00100.00 \pm 0.00
Complete-watermark recovery69.38

With the prespecified threshold of 82/128 matching bits, all 160 extractions pass ownership verification, including those with some incorrectly recovered bits.

2. CAST shows low false-positive rates in the tested ownership-verification settings. We test whether an unwatermarked model, or a model carrying a different watermark, is mistakenly accepted as the owner's model. Each setting uses 10,000 tests (Appendix D.8, Table 19):

TestFPR (%)
Unwatermarked model with a new random watermark, trigger, and key for each test0.13
Watermarked model with another watermark, trigger, and key0.09

Due to sampling randomness, we cannot guarantee a 0% FPR. The theoretical FPR of 0.09312% applies to a uniformly random watermark independent of the extracted result. The empirical results are close to the theoretical bound. The low FPRs complement the high ownership acceptance above, supporting reliable verification in these settings.


W2-1&Q2 Comparison with Memory-optimized EaaW

Thank you for suggesting this comparison!

CAST retains higher computation efficiency at comparable peak memory to optimized EaaW. Following the reviewer's suggestion, we implemented two-pass recomputation for EaaW: first compute the regression objective, then recompute model outputs to accumulate gradients without retaining all activation graphs. We added this comparison on 128-bit Qwen3-4B-Instruct-2507 in Appendix D.6 (Table 15) with a brief pointer in Section 3.5:

MethodPeak GPU memory (GiB)Time per pass (s)Gradient error vs. retained EaaW
EaaW, retained graphs84.893.544±0.1243.544 \pm 0.124Reference
EaaW, recomputation17.334.409±0.6194.409 \pm 0.619<1.2×10−7<1.2\times10^{-7}
CAST, sequential objectives18.052.910±0.0042.910 \pm 0.004-

Recomputation brings EaaW's memory usage close to CAST's, but requires additional forward passes. CAST processes its local objectives sequentially without this recomputation, using 34% less time at comparable peak memory. CAST therefore retains its computational advantage against memory-optimized EaaW.


AuthorsOpenReview ↗

Response to Reviewer Wyc5 (2/3)


W2-1&Q2 Comparison with Memory-optimized EaaW

Thank you for suggesting this comparison!

CAST retains higher computation efficiency at comparable peak memory to optimized EaaW. Following the reviewer's suggestion, we implemented two-pass recomputation for EaaW: first compute the regression objective, then recompute model outputs to accumulate gradients without retaining all activation graphs. We added this comparison on 128-bit Qwen3-4B-Instruct-2507 in Appendix D.6 (Table 15) with a brief pointer in Section 3.5:

MethodPeak GPU memory (GiB)Time per pass (s)Gradient error vs. retained EaaW
EaaW, retained graphs84.893.544±0.1243.544 \pm 0.124Reference
EaaW, recomputation17.334.409±0.6194.409 \pm 0.619<1.2×10−7<1.2\times10^{-7}
CAST, sequential objectives18.052.910±0.0042.910 \pm 0.004-

Recomputation brings EaaW's memory usage close to CAST's, but requires additional forward passes. CAST processes its local objectives sequentially without this recomputation, using 34% less time at comparable peak memory. CAST therefore retains its computational advantage against memory-optimized EaaW.


W2-2&Q2 Distinction from EaaW and Controlled Comparison

Thank you for asking us to clarify the distinction from EaaW!

1. CAST introduces token-wise attribution and vocabulary-subset observations to address two limitations of EaaW. EaaW also studies label-only verification for image classification, while CAST addresses text-only LLM APIs. We compare these two components below. Token-wise attribution replaces global regression with direct local differences, improving embedding efficiency. Vocabulary-subset observations turn generated tokens into a statistically estimable watermark signal for text-only verification.

2. Token-wise attribution improves the memory–time trade-off. CAST processes local objectives sequentially, whereas memory-optimized EaaW recomputes model outputs for its global regression objective. Appendix D.6 (Table 15) reports:

MethodPeak GPU memory (GiB)Time per pass (s)
EaaW, retained graphs84.893.544±0.1243.544 \pm 0.124
EaaW, recomputation17.334.409±0.6194.409 \pm 0.619
CAST, sequential objectives18.052.910±0.0042.910 \pm 0.004

CAST uses 34% less time at comparable peak memory to recomputation-enabled EaaW.

3. Vocabulary-subset observations enable accurate text-only watermark extraction. We equip EaaW's global regression with the same vocabulary-subset observations as CAST. The comparison below shows over 99% WSR for both subset-based methods, compared with 50% for the paper's original EaaW baseline:

MethodText-only WSR (%)
CAST99.53±1.5399.53 \pm 1.53
Original EaaW50.00
EaaW with subset observations99.84±0.7099.84 \pm 0.70

Together, these results support the complementary roles of CAST's two designs: token-wise attribution improves embedding efficiency, while vocabulary-subset observations support accurate text-only verification.


W2-3&Q4 API Query Cost

Thank you for highlighting verification costs!

1. Total token costs of CAST remain manageable, and requests can be spread out to meet API rate limits. CAST requests only one generated token per query. For 128 bits and T=100T=100, this gives approximately 12.9K generated tokens, plus 1.65M input tokens for 128-token prompts. These explicit token counts allow the cost to be budgeted, while requests can be scheduled over a longer period to stay within queries-per-minute (QPM) limits.

2. Fewer queries can still support successful ownership verification. We added a reduced-query evaluation on the same reference checkpoint in Appendix D.7 (Table 16):

TTTotal queriesRef. WSR (%)Ref. ownership acceptance (%)Local time (s)
101.3K92.34±4.6592.34 \pm 4.65100.0013.34±0.0613.34 \pm 0.06
253.2K98.55±2.2698.55 \pm 2.26100.0031.12±0.0231.12 \pm 0.02
10012.9K100.00±0.00100.00 \pm 0.00100.00118.90±0.36118.90 \pm 0.36

A 90% reduction in queries still supports successful ownership verification on the reference model.

3. Multi-token extraction could also reduce queries at the same generated-token budget. The multi-token extension introduced in our response to Reviewer C3c9’s W1&Q1 extracts multiple token scores per query, reducing queries at the same generated-token budget (Appendix D.7, Table 17):

Generated tokens per queryQueries per promptTotal queriesGenerated tokensInput tokens
1 (budget reference)9612.4K12.4K1.6M
2486.2K12.4K793K
4243.1K12.4K396K
6162.1K12.4K264K

At the same generated-token budget, four-token extraction reduces queries by 75% and also lowers input-token costs.


AuthorsOpenReview ↗

Response to Reviewer Wyc5 (3/3)


W2-4&Q4 Response Templates and Output Post-processing

Thank you for raising this practical concern! We describe the prefix-manipulation experiment in Appendix D.3 and report additional multi-token results in Appendix D.3.1 (Table 11).

1. CAST can recover the watermark under fixed, reversible API templates. Removing the known template restores watermark extraction when the original text and tokenization are preserved. Table 10 instead tests prefixes supplied before generation, including the simple prefix Sure.:

Prefix settingWSR (%)MMLU accuracy (%)
No attack100.0069.74
Short context prefix: Sure.59.0056.80
Long context prefix: OK, there is the answer.55.0056.16

Here, MMLU choices are scored immediately after the supplied prefix. A fixed prefix added after generation can preserve answer content and be removed before verification. Table 10 does not measure this case.

2. Templates that prevent reliable answer extraction also reduce API usability. If an attacker varies the template so that the verifier cannot reliably separate it from the generated answer, ordinary applications using the same parsing procedure face the same difficulty.

3. CAST can also be extended to multi-token embedding and verification, overcoming reliance on the first token alone. We extended CAST to embed and extract the watermark across 2, 4, or 6 output tokens, then tested whether it remains recoverable after deleting the first token:

Generated tokens per queryQueries per promptFull-text WSR (%)WSR after first-token deletion (%)
248100.0053.91
424100.00100.00
61697.6697.66

The four- and six-token versions still pass ownership verification after deletion (threshold: 82/128). This demonstrates a feasible way to mitigate first-token dependence.

Reviewer_Nik4OpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CAST, an LLM model copyright auditing method. CAST embeds watermark bits into the differences between vocabulary-subset probabilities for an original trigger prompt and token-masked prompts. The watermark signal is recovered via frequency-based statistical estimation, enabling verification without logits and sequential embedding with reduced peak auxiliary memory. The paper also conducted extensive experiments to compare CAST with different baselines.

Critical strengths
  1. The LLM model copyright auditing problem is important and timely. Section 2 (Background) provides a concise and useful literature review in this area.
  2. The CAST design is well motivated by EaaW. The CAST design is simple and interesting.
  3. The experiment is comprehensive.
Critical weaknesses
  1. Robustness and vulnerability issue: CAST measures the probability distribution of the first generated token after the raw trigger X. This is a fundamental weakness of the method. For example, adding a short fixed prefix reduces CAST's WSR to 59% (close to a random guess, 50%) as shown in Appendix D.3 (Table 10). While the authors argue that forcing a prefix degrades MMLU by around 12%, in practice a short neutral prefix ("Sure... ") would probably cost little real utility.

  2. Practical issue: To extract a 128 bit watermark with T = 100, CAST requires submitting T(k+1) = 12,900 very similar queries (only one token difference). In reality, even the simplest API rate-limiter or prompt-similarity filter would immediately block this probe.

Questions for authors
  1. The Figure 1 caption uses V_1 while the diagram uses V_e. Are they the same or different?
  2. The fine-tuning attack setup (Appendix D.3, Table 9) is not specified: LoRA or full-parameter, learning rate, number of examples? Please provide more details.
4 comments
AuthorsOpenReview ↗

Response to Reviewer Nik4

Dear Reviewer Nik4,

Thank you for your detailed comments and constructive suggestions! We appreciate your recognition of the importance of LLM copyright auditing, CAST's simple and motivated design, and the breadth of our experiments. We address your concerns below.


W1 First-token Dependence and Prefix Manipulation

Thank you for raising this practical concern! We describe the prefix-manipulation experiment in Appendix D.3 and report additional multi-token results in Appendix D.3.1 (Table 11).

1. CAST can recover the watermark under fixed, reversible API templates. Removing the known template restores watermark extraction when the original text and tokenization are preserved. Table 10 instead tests prefixes supplied before generation, including the simple prefix Sure.:

Prefix settingWSR (%)MMLU accuracy (%)
No attack100.0069.74
Short context prefix: Sure.59.0056.80
Long context prefix: OK, there is the answer.55.0056.16

Here, MMLU choices are scored immediately after the supplied prefix. Fixed prefixes added after generation can instead be removed before watermark verification.

2. Templates that prevent reliable answer extraction also reduce API usability. If an attacker varies the template so that the verifier cannot reliably separate it from the generated answer, ordinary applications using the same parsing procedure face the same difficulty.

3. CAST can also be extended to multi-token embedding and verification, overcoming reliance on the first token alone. We extended CAST to embed and extract the watermark across 2, 4, or 6 output tokens, then tested whether it remains recoverable after deleting the first token:

Generated tokens per queryQueries per promptFull-text WSR (%)WSR after first-token deletion (%)
248100.0053.91
424100.00100.00
61697.6697.66

The four- and six-token versions still pass ownership verification after deletion (threshold: 82/128). This demonstrates a feasible way to mitigate first-token dependence.


W2 Query Budget and API Restrictions

Thank you for highlighting verification costs!

1. Total token costs of CAST remain manageable, and requests can be spread out to meet API rate limits. CAST requests only one generated token per query. For 128 bits and T=100T=100, this gives approximately 12.9K generated tokens. Spreading these queries over time lets the verifier meet queries-per-minute (QPM) limits.

2. Fewer queries can still support successful ownership verification. We added a reduced-query evaluation on the same reference checkpoint in Appendix D.7 (Table 16):

Samples per prompt TTTotal queriesWSR (%)Ownership acceptance (%)
101.3K92.34±4.6592.34 \pm 4.65100.00
253.2K98.55±2.2698.55 \pm 2.26100.00
10012.9K100.00±0.00100.00 \pm 0.00100.00

A 90% reduction in queries still supports successful ownership verification on the reference model.

3. Multi-token extraction could also reduce queries at the same generated-token budget. The multi-token extension introduced in our response to Reviewer C3c9’s W1&Q1 extracts multiple token scores per query, reducing queries at the same generated-token budget (Appendix D.7, Table 17):

Generated tokens per queryQueries per promptTotal queriesGenerated tokens
1 (budget reference)9612.4K12.4K
2486.2K12.4K
4243.1K12.4K
6162.1K12.4K

At the same generated-token budget, four-token extraction reduces queries by 75% and also lowers input-token costs.

4. Long-term prompt-similarity filtering may incur substantial overhead at high request volumes. Spreading verification queries over time could reduce bursts of similar prompts. To detect matches over longer intervals, an account-wide filter must also screen ordinary requests, store representations of past prompts, and search that history as new requests arrive. This may lead to substantial overhead for the attacker, making this type of attack infeasible.


Q1 Notation in Figure 1

Thank you for catching this inconsistency!

V1V_1 and VeV_e denote the same explaining vocabulary subset. We corrected the Figure 1 caption to use VeV_e consistently.


Q2 Fine-tuning Attack Settings

Thank you for requesting this clarification!

The fine-tuning attack updates the existing watermarked LoRA adapter while freezing the base model. We specify its configuration in Appendix D.3.

AuthorsOpenReview ↗

Thanks to Reviewer Nik4

Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the importance of LLM copyright auditing, CAST's simple and motivated design, and the breadth of our experiments.

Please let us know if our response and the new experiments have properly addressed your concerns. We are more than happy to answer any additional questions during the discussion period. Your feedback will be greatly appreciated.

Thanks authors for the response. However, the response did not clarify my concern about the fundamental weakness and practical issue well. I will reduce my evaluation score to 1.

Dear Reviewer Nik4,

Thank you for your feedback.

  1. About Robustness: We showed that even Sure. substantially harms safety and instruction following, limiting its practical value as a removal attack (see our follow-up response to Reviewer Wyc5).
  2. About Practical Issues: We demonstrated query-reduction extensions and discussed request scheduling to meet QPM limits and mitigate prompt filtering.

Could you please clarify which concerns remain unresolved? We would be happy to provide further explanations.

Reviewer_C3c9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes CAST, a scalable LLM model watermarking framework for text-only copyright auditing. CAST embeds watermark bits into token-wise attribution scores by manipulating probability shifts over a secret vocabulary subset. Its independent token-wise optimization reduces memory overhead, while frequency-based estimation enables watermark extraction without access to logits. Experiments across multiple LLMs demonstrate high watermark recovery rates, minimal utility degradation, and robustness against common model modifications.

Critical strengths
  1. Novel and effective watermarking mechanism. The proposed token-wise attribution mechanism offers an interesting approach to model watermarking. In particular, using probability shifts over a secret vocabulary subset enables reliable text-only watermark extraction without requiring access to logits. This addresses an important limitation of previous explanation-based watermarking methods.
  2. Strong experimental results with good scalability and fidelity. CAST achieves near-perfect watermark recovery rates across multiple LLMs and different auditing scenarios while largely preserving the original model utility. The experiments also demonstrate improved memory efficiency over EaaW and robustness against common modifications such as fine-tuning and quantization.
Critical weaknesses
  1. Limited evaluation under realistic API conditions. CAST's text-only extraction relies on statistical estimation from the first generated token. Although the paper evaluates different decoding temperatures and prefix manipulation attacks, its robustness to common API-level output transformations, such as response templates or post-processing, remains unclear, particularly when these transformations preserve the model's underlying utility.
  2. The scalability advantage could be further investigated. While Table 13 demonstrates substantial memory savings over EaaW, the comparison is limited to the implementations evaluated in the paper. It would be helpful to clarify whether memory optimization techniques, such as gradient recomputation, could reduce this gap. In addition, text-only verification requires a relatively large number of API queries, and its practical overhead could be better characterized.
Questions for authors
  1. Appendix D.3 shows that prefix manipulation significantly reduces the WSR while also degrading MMLU performance. Would the same utility degradation occur if an API provider applied a fixed response template through post-processing, without modifying the model's actual generation? How robust would CAST be under such conditions?
  2. Theorem 1 assumes independence across watermark-bit recovery events. However, Algorithm 2 reuses the same empirical estimate of PVe(X)P_{V_e}(X) for all watermark bits, potentially introducing correlations between their recovery outcomes. Could the authors justify this independence assumption or provide a theoretical bound that accounts for such dependencies?
4 comments
AuthorsOpenReview ↗

Response to Reviewer C3c9 (1/2)

Dear Reviewer C3c9,

Thank you for your thoughtful comments and constructive suggestions! We appreciate your recognition of CAST's text-only verification mechanism, scalability, and preservation of model utility. We address your concerns below.


W1&Q1 Response Templates and Output Post-processing

Thank you for raising this practical concern! We describe the prefix-manipulation experiment in Appendix D.3 and report additional multi-token results in Appendix D.3.1 (Table 11).

1. CAST can recover the watermark under fixed, reversible API templates. Removing the known template restores watermark extraction when the original text and tokenization are preserved. Table 10 instead tests a prefix supplied to the model before generation.

2. Templates that prevent reliable answer extraction also reduce API usability. If an attacker varies the template so that the verifier cannot reliably separate it from the generated answer, ordinary applications using the same parsing procedure face the same difficulty.

3. CAST can also be extended to multi-token embedding and verification, overcoming reliance on the first token alone. We extended CAST to embed and extract the watermark across 2, 4, or 6 output tokens, then tested whether it remains recoverable after deleting the first token:

Generated tokens per queryQueries per promptFull-text WSR (%)WSR after first-token deletion (%)
248100.0053.91
424100.00100.00
61697.6697.66

The four- and six-token versions still pass ownership verification after deletion (threshold: 82/128). This demonstrates a feasible way to mitigate first-token dependence.


W2-1 Comparison with Memory-optimized EaaW

Thank you for suggesting this comparison!

CAST retains higher computation efficiency at comparable peak memory to optimized EaaW. Following the reviewer's suggestion, we implemented two-pass recomputation for EaaW: first compute the regression objective, then recompute model outputs to accumulate gradients without retaining all activation graphs. We added this comparison on 128-bit Qwen3-4B-Instruct-2507 in Appendix D.6 (Table 15) with a brief pointer in Section 3.5:

MethodPeak GPU memory (GiB)Time per pass (s)
EaaW, retained graphs84.893.544±0.1243.544 \pm 0.124
EaaW, recomputation17.334.409±0.6194.409 \pm 0.619
CAST, sequential objectives18.052.910±0.0042.910 \pm 0.004

Recomputation brings EaaW's memory usage close to CAST's, but requires additional forward passes. CAST processes its local objectives sequentially without this recomputation, using 34% less time at comparable peak memory. CAST therefore retains its computational advantage against memory-optimized EaaW.


W2-2 API Query Cost

Thank you for highlighting verification costs!

1. Total token costs of CAST remain manageable, and requests can be spread out to meet API rate limits. CAST requests only one generated token per query. For 128 bits and T=100T=100, this gives approximately 12.9K generated tokens. These explicit token counts allow the cost to be budgeted, while requests can be scheduled over a longer period to stay within queries-per-minute (QPM) limits.

2. Fewer queries can still support successful ownership verification. We added a reduced-query evaluation on the same reference checkpoint in Appendix D.7 (Table 16):

Samples per prompt TTTotal queriesWSR (%)Ownership acceptance (%)
101.3K92.34±4.6592.34 \pm 4.65100.00
253.2K98.55±2.2698.55 \pm 2.26100.00
10012.9K100.00±0.00100.00 \pm 0.00100.00

A 90% reduction in queries still supports successful ownership verification on the reference model.

3. Multi-token extraction could also reduce queries at the same generated-token budget. The multi-token extension introduced in our response to Reviewer C3c9’s W1&Q1 extracts multiple token scores per query, reducing queries at the same generated-token budget (Appendix D.7, Table 17):

Generated tokens per queryQueries per promptTotal queriesGenerated tokens
1 (budget reference)9612.4K12.4K
2486.2K12.4K
4243.1K12.4K
6162.1K12.4K

At the same generated-token budget, four-token extraction reduces queries by 75% and also lowers input-token costs.

AuthorsOpenReview ↗

Response to Reviewer C3c9 (2/2)


Q2 Dependence-aware Recovery Analysis

Thank you for pointing out the dependence between bit estimates!

1. Bit recovery is independent once the shared original-prompt estimate is fixed. Given this estimate, independently sampled masked-prompt responses yield independent bit decisions. We can therefore multiply their conditional recovery probabilities and then average over the shared estimate, without changing CAST or adding queries.

2. We can also bound recovery without assuming bit independence. Keeping Theorem 1's Gaussian approximation and single-bit analysis, a union bound gives

Pr⁡(w^=w)≥max⁡(0,1−k[1−Φ(ϵ2T1−ϵ2)]).\Pr(\hat{w} = w) \geq \max (0, 1 - k[1 - \Phi(\epsilon \sqrt{\frac{2T}{1-\epsilon^2}})]).

This replaces the product of marginal success probabilities and allows dependent bit-recovery events. We revised Theorem 1 in Section 3.5 and its proof in Appendix A.

3. Removing the independence assumption gives a similar watermark-recovery lower bound. With the same k=128k=128, T=100T=100, and ϵ=0.25\epsilon=0.25:

AnalysisComplete-recovery lower bound (%)
Original product, assuming independent bit recovery98.345
Union bound, without assuming independent bit recovery98.331

The two bounds differ by only 0.014 percentage points, retaining a similar recovery bound without the independence assumption.

Thank you for your experiment and clarification, my concern has been resolved. I'll keep my overall score(3 Weak Accept).

Dear Reviewer C3c9,

Thank you for your thoughtful feedback and positive assessment! We are glad that our responses have addressed your concerns, and we deeply appreciate your time and constructive suggestions :)

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal and Revision Summary

Dear AC and Reviewers,

Thank you for your thoughtful reviews and constructive suggestions! Your insightful comments really help us improve our manuscript. We have carefully addressed all key concerns through revised theoretical analysis, targeted experiments, and clearer explanations. These additions strengthen the theoretical and empirical support for CAST, as summarized below.

  • The recovery bound holds without assuming bit independence. We revised Theorem 1 using a union bound, retaining an approximate recovery lower bound under the Gaussian approximation (Section 3.5 and Appendix A).

  • Multi-token CAST mitigates first-token dependence. We extended CAST to multi-token embedding and extraction. The four- and six-token variants still pass ownership verification after first-token deletion (Appendix D.3.1, Table 11). We also clarify in the rebuttal how fixed, reversible API templates can be removed before verification.

  • CAST remains more efficient than recomputation-optimized EaaW. Our comparison shows 34% less watermark-objective computation time at comparable peak GPU memory (Appendix D.6, Table 15).

  • Ownership verification can use substantially fewer queries. The reference model passes all twenty extractions with 90% fewer queries. Four-token verification provides another route, reducing queries by 75% at the same generated-token budget (Appendix D.7, Tables 16–17).

  • Repeated extraction is stable, with low observed false-positive rates. All 160 extractions from eight successfully watermarked models pass ownership verification. False-positive rates are 0.13% and 0.09% over 10,000 tests per setting (Appendix D.8, Tables 18–19).

We have also clarified the fine-tuning attack configuration in Appendix D.3 and corrected the vocabulary-subset notation in Figure 1. We have provided the revised manuscript, with revisions highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.