acceptodds
Under review as a conference paper at ICLR 2027

Language Model Fingerprinting Requires Rethinking Watermark Teachers

29 views

Abstract

LLM fingerprinting via watermark distillation embeds a statistical watermark signal into model weights, enabling model owners to identify their models behind black-box APIs. Revisiting a recent protocol, we find that its utility evaluation understates text quality degradation in open-ended generation, favoring overly strong watermark teachers. Weakening the watermark improves text quality but sacrifices detectability. To move beyond this trade-off, we rethink whether text watermarks designed for verifying generated text are suitable distillation teachers for model fingerprinting. Such watermarks are typically designed to remain detectable from an individual output, limiting how sparse the watermark signal can be. In contrast, fingerprint verification can aggregate signal across queries, making sparser watermark signals viable. This raises a key question: where should the sparse signal be placed? We analyze signal placement through token surprisal and show that, even at comparable watermark strength, different placements can target tokens with different plausibility under the base model. This motivates near-tie restriction, which uses top-1-relative logit gaps to restrict the watermark bias to tokens close to the base model's top prediction. Across multiple models, near-tie improves detection–quality frontiers under deployment changes, preserves higher text quality across query budgets, and further improves existing watermarking schemes when combined with them.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM fingerprint, watermark

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ics1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an improvement over watermark-based fingerprinting method. They show that on the fingerprint domain, the watermark significantly degrades the generated text quality and that lowering the watermarking strength can improve quality but decreases the fingerprint detectability. They find that, in practice, the fingerprint does not need to use the full watermark, and that restricting the watermark to only the most likely tokens keep a strong fingerprinting signal without significantly impacting quality.

Critical strengths
  • The idea of using a "weaker" watermark teacher for fingerpriting is intuitive, and the proposed near-tie teacher appears to be very effective at preserving the fingerprint detectability while limiting the impact on model quality.
  • The paper is very easy to follow and the evaluation is thorough.
Critical weaknesses
  • The novelty of this work is quite limited. This work iteratively improves the method established in [1], and the implication "aggregating signals across queries make sparser signals viable" seems to be similar to the idea in [1] of reducing the watermark strength δ\delta. The framing of this work is also a bit aggressive. Even though the authors argue the evaluation protocol in [1] favors overly strong teacher, it seems the key issue is a mislabelling of a given metric (which should be reported to the authors of [1]) and that the "normal" watermark distillation induces too high distortion.
  • The near-tie watermark intuitively seems to have a similar effect as lowering the temperature (because it increases the student probability of sampling a high probability teacher token) which mechanistically increases the PPL. In Table 4, we see that, for some models, the PPL of the fingerprinted KGW-NT model is lower than the base model, but (assuming the base model has the "best" quality) that suggests still a quality degradation. A better metric might be the L1 norm of the base model PPL and the fingerprinted model PPL.

[1] LLM fingerprinting via semantically conditioned watermarks, Gloaguen et al., ICLR 2026

Questions for authors
  • As the authors mentioned, a watermark-based fingerprint only needs a sparse signal to be detected. The proposed method still embeds the watermark in every token (eventhough restricted to the top logits). Have the authors also explored more deviation from the KGW watermarks? I believe exploring to what extend the watermark needs to be embedded is an important direction.
  • Could the authors show that the lower PPL translates in low quality impact and is not only an artifact of decreasing temperature? Maybe metrics like pass@k on hard benchmarks could distinguish such cases?
4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

We appreciate the reviewer's recognition of our method's effectiveness and the clarity of our presentation. However, we are concerned that some of the comments may stem from misunderstandings of our key contributions and methodology. We clarify these points below and have incorporated the corresponding revisions into the manuscript, highlighted in pink.

W1. The novelty of this work is quite limited. This work iteratively improves the method established in [1], and the implication "aggregating signals across queries make sparser signals viable" seems to be similar to the idea in [1] of reducing the watermark strength. The framing of this work is also a bit aggressive. Even though the authors argue the evaluation protocol in [1] favors overly strong teacher, it seems the key issue is a mislabelling of a given metric (which should be reported to the authors of [1]) and that the "normal" watermark distillation induces too high distortion.

Q1. As the authors mentioned, a watermark-based fingerprint only needs a sparse signal to be detected. The proposed method still embeds the watermark in every token (eventhough restricted to the top logits). Have the authors also explored more deviation from the KGW watermarks? I believe exploring to what extend the watermark needs to be embedded is an important direction.

Our contribution goes beyond correcting a mislabeled metric or simply weakening the KGW watermark teacher. While [1] establishes an effective fingerprinting framework, its utility evaluation primarily relies on accuracy-based benchmarks and reports mean token entropy as PPL, overlooking substantial quality degradation in open-ended generation (Table 1). Importantly, even under its multi-query verification protocol, reducing the KGW bias strength does not resolve the detection–quality trade-off: Figure 2 shows that δ=2\delta=2 still more than doubles PPL, whereas δ=1\delta=1 falls below the detection threshold.

To overcome this trade-off, we reconsider whether conventional text watermark teachers are well suited for model fingerprinting. Unlike text watermarking, which typically requires detection from individual outputs, fingerprint verification aggregates watermark signal across queries, allowing sparser signals. Yet [1] retains the conventional KGW teacher that biases every green token, without exploiting this distinction in teacher design.

This leads us to investigate where to place the sparse watermark signal, rather than merely how much to weaken it. Our token-surprisal analysis (Proposition 5.1) shows that, even at matched watermark strength, biasing more plausible tokens induces a smaller surprisal shift. Guided by this insight, we introduce near-tie restriction, which biases only green tokens close to the top-1 logit. Unlike conventional KGW, it does not apply bias at every token: if no green token satisfies the near-tie condition, no bias is applied (Section 5.3). Since near-tie restriction simply limits the set of biased green tokens, it can also be combined with other watermarking schemes.

Empirically, KGW-NT achieves a better detection–quality trade-off than the baselines, including KGW-TK, which restricts the bias to top-k tokens (Figure 3). This highlights the importance of logit-gap-based signal placement beyond simply restricting the biased token set. We further examine the degree of restriction by varying τ\tau across the 10th, 30th, 50th, and 70th percentiles of training-data logit gaps, finding favorable detection–quality trade-offs at the 30th–50th percentiles (Figure 5). Furthermore, applying near-tie restriction to SWEET and MorphMark also improves their detection–quality trade-offs (Figure 6), demonstrating its compatibility across different watermarking schemes.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W2. The near-tie watermark intuitively seems to have a similar effect as lowering the temperature (because it increases the student probability of sampling a high probability teacher token) which mechanistically increases the PPL. In Table 4, we see that, for some models, the PPL of the fingerprinted KGW-NT model is lower than the base model, but (assuming the base model has the "best" quality) that suggests still a quality degradation. A better metric might be the L1 norm of the base model PPL and the fingerprinted model PPL.

Q2. Could the authors show that the lower PPL translates in low quality impact and is not only an artifact of decreasing temperature? Maybe metrics like pass@k on hard benchmarks could distinguish such cases?

To examine whether temperature-based sharpening can provide the gains of KGW-NT, we extend our original temperature analysis on Llama-3.1-8B to all baselines and additionally evaluate Qwen2.5-3B, sweeping the decoding temperature T∈0.1,0.3,0.5,0.7,1.0T \in \\{0.1, 0.3, 0.5, 0.7, 1.0\\} while keeping KGW-NT fixed at T=1.0T=1.0. The revised Figure 13 shows that KGW-NT maintains a favorable detection–quality Pareto frontier even when the baselines are allowed to tune their decoding temperatures. These results indicate that temperature tuning alone does not provide KGW-NT's gains in this setting.

Regarding PPL ratios below 1, the base model's PPL should not be treated as the optimal target. Since PPL is measured using an external reference model, a PPL ratio below 1 does not imply quality degradation. Likewise, the absolute PPL difference is not necessarily a better quality metric, as matching the base model's PPL does not guarantee better text quality.

To complement PPL, we also evaluate linguistic quality using an LLM judge that assesses grammatical correctness, fluency, coherence, readability, lexical appropriateness, and repetition (judge prompt provided in Appendix J). Figures 3 (right) and 7 (bottom) show no apparent quality degradation relative to the base models in settings where PPL ratios fall below 1.

To further assess these settings, we evaluate mathematical reasoning on 250 French MGSM problems using Qwen2.5-3B, with 16 samples per problem at T=1.0T=1.0 and top-p=0.9p=0.9. The table below (also Table 6 in the revised manuscript) shows that KGW-NT maintains pass@k performance comparable to the base model despite its lower PPL. In particular, KGW-NT(q=50q=50) achieves 73.1% pass@1 and 96.8% pass@16, compared with 71.5% and 97.2% for the base model, while also achieving a higher LLM judge score (7.66 vs. 7.27). These results provide further evidence that lower PPL does not necessarily come at the expense of linguistic quality or reasoning performance. We also confirm that KGW-NT preserves downstream utility on the summarization task (Appendix I.8 and Table 5).

MethodPPL ratio ↓Worst-z ↑Judge ↑Pass@1 ↑Pass@4 ↑Pass@16 ↑
Base1.00—7.2771.5090.2097.20
KGW (δ=2\delta=2)1.278.666.8168.3089.2096.80
SWEET1.026.377.3270.6089.6096.00
MorphMark1.027.307.2171.0089.3095.20
KGW-TK0.877.527.2669.8089.3095.60
KGW-NT (q=30q=30)0.786.667.5672.3090.2096.80
KGW-NT (q=50q=50)0.799.317.6673.1090.6096.80

Thank you for your reply and additional experiments, however I still have some remaining concerns.

  • I believe I have not misunderstood your key contributions and methodology. My concern was about the depth of your investigation. In its current form, the method proposed is quite narrow and I think this work would benefit from deeper deviation from [1]'s method.
  • Thanks for the additional experiment, it confirms that the method is effective at fingerprinting. My concerns in that regard have been addressed. Overall, the authors rebuttal experiments confirm my review: the proposed method is sound and indeed improves the fingerprinting performance (better quality-detectability trade-off) compare to [1]. However, I still find the investigation limited with little informativeness. Thus, I will keep my score.
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for acknowledging that our method is sound, improves the detection–quality trade-off over [1], and that our additional experiments have addressed your concerns regarding its effectiveness.

While you state that you have not misunderstood our key contributions or methodology, our concern about potential misunderstandings was based on specific factual inaccuracies and mischaracterizations in the original review. We addressed these points in our earlier rebuttal and briefly reiterate them below.

Q1. "The proposed method still embeds the watermark in every token"

This is factually incorrect. KGW-NT applies no watermark bias at positions where no green token satisfies the near-tie condition (Section 5.3).

W1. "aggregating signals across queries make sparser signals viable" seems to be similar to the idea in [1] of reducing the watermark strength δ\delta.

Simply weakening the watermark strength δ\delta does not resolve the detection–quality trade-off: Figure 2 shows that δ=2\delta=2 still more than doubles PPL, whereas δ=1\delta=1 falls below the detection threshold. Instead, guided by Proposition 5.1, we selectively restrict which tokens receive watermark bias rather than reducing the bias strength δ\delta.

Q1. "Have the authors also explored more deviation from the KGW watermarks?"

We have already extended the near-tie restriction to SWEET and MorphMark, demonstrating improved detection–quality trade-offs (Figure 6). Compatibility with existing watermarking schemes is an important strength, particularly given the diversity of advanced watermarking methods already available.

W1. "it seems the key issue is a mislabelling of a given metric"

Our contribution goes beyond correcting a mislabeled metric. We uncover substantial open-ended generation quality degradation overlooked by [1]'s evaluation protocol (Table 1), revealing that the prior evaluation favors overly strong watermark teachers.

We therefore respectfully disagree with your assessment that our investigation is limited and offers little new insight. Beyond the proposed method, we identify overlooked quality degradation in existing fingerprinting evaluations and provide theoretical and empirical insights into sparse watermark signal placement. Our broader finding is that watermark teachers should be designed specifically for model fingerprinting, and our analysis provides concrete guidance for doing so. We believe our contributions should be assessed based on the new findings and insights they provide, rather than solely on the degree of methodological deviation from [1].

Reviewer_o4xAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper revisits watermark-based LLM fingerprinting (Gloaguen et al., 2026), where a KGW watermark teacher is distilled into model weights on a secret domain (French). It first shows that the prior protocol understates quality loss in open-ended generation: its reported “PPL” is actually mean token entropy, and the default δ=4 teacher raises real PPL from 4.63 to 28.27 on Llama-3.1-8B (Table 1). Because verification aggregates evidence across many queries, the authors argue that a sparser signal is enough. They propose near-tie restriction, which applies the bias only to green tokens close to the base model’s top-1 logit and scores only the matching positions at detection time. Across four models, KGW-NT gives a better worst-case detectability vs. PPL-ratio frontier than KGW, SWEET, MorphMark and KGW-TK, and near-tie also improves SWEET and MorphMark when combined with them.

Critical strengths
  1. The re-evaluation of Gloaguen et al. is useful on its own. It shows that the existing protocol mislabels entropy as PPL and that its default teacher seriously degrades open-ended generation for users in the fingerprint domain.

  2. The method is simple and can be added to existing watermarking schemes. It also has sensible controls: a comparison against random placement at matched strength, and an empirical check that the detector’s false-positive rate stays calibrated.

Critical weaknesses
  1. Robustness rests on one mild fine-tuning setting: In Table 5, fine-tuning (FT) is the binding worst case for every method. For KGW-NT (q=30) on Llama-3.1-8B, z falls from 17.30 to 5.36, barely above ρ=4 and below SWEET (6.57), MorphMark (6.38) and KGW-TK (5.55). The best PPL points on the frontier therefore sit just above threshold after a single epoch of French WildChat at lr 2e-5. The paper’s premise is a sparser signal, and the model’s weights are open, so the central untested question is how fast that signal disappears under stronger in-domain FT (more epochs or data, LoRA, distillation).

  2. Gains are confounded with the detector and with sharpening: KGW-NT uses the base-model near-tie detector, while KGW-TK and MorphMark use the standard KGW detector. Figure 12 separates the detector’s effect only against KGW, not against KGW-TK, the closest competitor. KGW-NT also lowers entropy (H 0.93 vs. 1.12 for the base model on 8B) and gives PPL ratios of 0.78–0.85 on the 3B models with higher self-BLEU. External-model PPL and a fluency-only judge (Appendix J) both reward that. The temperature sweep (Figure 13) compares only against KGW and without deployment transformations.

Questions for authors
  1. How do KGW-TK, SWEET and MorphMark perform when scored with an equivalent base-model position filter, i.e., Figure 12 extended to all baselines?

  2. What are the worst-case z-scores of KGW-NT (q=30/50) and KGW (δ=2) after stronger in-domain FT, such as 3 epochs, more WildChat data, or LoRA?

  3. Does the temperature-matched comparison in Figure 13 still hold against KGW-TK and SWEET, and under transformations?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

We thank the reviewer for their feedback. We address each comment below and have incorporated the corresponding revisions into the manuscript, highlighted in pink.

W1. Robustness rests on one mild fine-tuning setting: In Table 5, fine-tuning (FT) is the binding worst case for every method. For KGW-NT (q=30) on Llama-3.1-8B, z falls from 17.30 to 5.36, barely above ρ=4 and below SWEET (6.57), MorphMark (6.38) and KGW-TK (5.55). The best PPL points on the frontier therefore sit just above threshold after a single epoch of French WildChat at lr 2e-5. The paper’s premise is a sparser signal, and the model’s weights are open, so the central untested question is how fast that signal disappears under stronger in-domain FT (more epochs or data, LoRA, distillation).

Q2. What are the worst-case z-scores of KGW-NT (q=30/50) and KGW (δ=2) after stronger in-domain FT, such as 3 epochs, more WildChat data, or LoRA?

Our original robustness evaluation follows the protocol of prior work [1]. To examine stronger fine-tuning, we additionally evaluate Llama-3.1-8B with 3 training epochs, 1.63× more French WildChat data (using all remaining available data), and LoRA at a learning rate of 2×10−52\times10^{-5}. We apply the same settings to all baselines.

The table below (also Table 4 in the revised manuscript) shows that increasing the number of training epochs or the amount of training data reduces detectability across methods, while LoRA has a smaller impact. After 3 epochs, KGW-NT (q=30q=30) and KGW (δ=2\delta=2) fall below the detection threshold (z=4z=4), whereas KGW-NT (q=50q=50) remains detectable (z=6.26z=6.26) and outperforms all baselines. With additional training data, all methods remain detectable. Notably, KGW-NT (q=50q=50) achieves the highest robustness across all evaluated settings while maintaining a PPL ratio close to the base model (1.04). Notably, these experiments assume fine-tuning directly on the secret fingerprint domain, which is typically unknown to the adversary.

MethodPPL ratio ↓FT 1 ep ↑FT 3 ep ↑FT 1.63× data ↑LoRA 2e-5 ↑
KGW (δ=2\delta=2)2.168.053.755.9819.21
SWEET1.546.574.095.1413.93
MorphMark1.456.382.775.1613.32
KGW-TK1.205.552.444.6112.83
KGW-NT (q=30q=30)1.025.362.584.9214.40
KGW-NT (q=50q=50)1.0410.056.268.3522.26

W2-1. Gains are confounded with the detector and with sharpening: KGW-NT uses the base-model near-tie detector, while KGW-TK and MorphMark use the standard KGW detector. Figure 12 separates the detector’s effect only against KGW, not against KGW-TK, the closest competitor.

Q1. How do KGW-TK, SWEET and MorphMark perform when scored with an equivalent base-model position filter, i.e., Figure 12 extended to all baselines?

To isolate the effect of detector choice, we extend Figure 12 by applying the same near-tie detector to all baselines (KGW, KGW-TK, SWEET, and MorphMark) on Llama-3.1-8B. We keep their watermark teachers unchanged and sweep the near-tie threshold (τ\tau) over the same values used for KGW-NT.

The revised Figure 12 shows that KGW-NT maintains a favorable detection–quality Pareto frontier, particularly at low PPL ratios, even when the baselines use the near-tie detector. KGW-NT also retains a favorable trade-off when evaluated with the standard KGW detector, demonstrating that its gains are not solely attributable to detector choice.

W2-2. KGW-NT also lowers entropy (H 0.93 vs. 1.12 for the base model on 8B) and gives PPL ratios of 0.78–0.85 on the 3B models with higher self-BLEU. External-model PPL and a fluency-only judge (Appendix J) both reward that.

While KGW-NT reduces output entropy and slightly increases self-BLEU, the increase in self-BLEU is modest (0.02–0.05 across models), and Distinct-2/3 remains nearly unchanged (Table 7 in the revised manuscript). We note that our LLM judge complements PPL by assessing broader aspects of linguistic quality beyond fluency, including grammatical correctness, coherence, readability, lexical appropriateness, and repetition (Appendix J).

KGW-NT also achieves better text quality even at comparable entropy and self-BLEU. On Llama-3.1-8B, KGW-TK and KGW-NT (q=30) have similar entropy (1.05 vs. 1.03) and self-BLEU (0.18 vs. 0.19), yet KGW-NT achieves a lower PPL ratio (1.02 vs. 1.20) and a higher judge score (7.40 vs. 7.06). These results suggest that the choice of biased tokens, rather than output entropy alone, drives the quality differences.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W2-3.The temperature sweep (Figure 13) compares only against KGW and without deployment transformations.

Q3. Does the temperature-matched comparison in Figure 13 still hold against KGW-TK and SWEET, and under transformations?

We clarify that Figure 13 keeps KGW-NT fixed at T=1.0T=1.0 while tuning the baselines' decoding temperatures to examine whether temperature tuning can provide KGW-NT's gains, rather than comparing methods at matched temperatures. We extend this analysis to all baselines (KGW, SWEET, MorphMark, and KGW-TK) and evaluate worst-case detection z-scores across deployment transformations on Llama-3.1-8B. The revised Figure 13 shows that KGW-NT maintains a favorable detection–quality Pareto frontier, particularly at low PPL ratios, even when the baselines are allowed to tune their temperatures. These results indicate that temperature tuning alone does not provide KGW-NT's gains in this setting, highlighting the importance of selectively applying watermark bias to plausible near-tie tokens.

Thank you for the clarifications and additional experiments. The new results address my concerns about fine-tuning robustness, detector choice, and temperature tuning. I will keep my original positive rating.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your positive feedback and for taking the time to review our additional experiments. We appreciate your consideration and are glad that the new results addressed your concerns.

Reviewer_4KSvOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

In this work, the authors focus on watermark-based fingerprinting, which uses a text watermark to verify the provenance of generated text as a distillation teacher. By revisiting the recent evaluation protocol in this field, the authors find the existence of a trade-off relationship between detectability and text quality in watermarking. Furthermore, to deal with problem, the authors also propose a new method, near-tie restriction, which limits teacher modifications to tokens close to the model’s top prediction. Experimental results of various large language models (LLMs) show the improved detectability of near-tie restriction while keeping the fluency of the target text.

Critical strengths
  • This work reveals that recent watermark-based fingerprinting evaluations understate text quality degradation while focusing on detectability.
  • In addition, the trade-off relationship between text quality and detectability is revealed.
  • Furthermore, the authors proposed near-tie restriction, a solution to the problem.
  • The experimental results of various LLMs show the detectability improvement of the proposed method with keeping the fluency of the text.
Critical weaknesses
  • Text quality is mostly judged by perplexity scores. However, it only measures fluency. For reliable evidence, the authors should have checked the text quality on some downstream tasks like summarization.
  • The perplexity is calculated by Qwen-2.5-32B. Given that the targeted LLMs include Qwen-2.5 variants, this setting is biased and unfair to Llama-3 models.
  • The proposed method requires tuning the hyperparameter τ\tau. Given that tuning the threshold of epsilon sampling can improve text generation performance in natural language generation fields, readers cannot judge whether text quality is improved by the proposed method itself or by improved sampling through careful hyperparameter tuning.
Questions for authors
  • What is the reason for using Qwen2.5 rather than Qwen3.x?
  • How did you decide on each hyperparameter?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for their feedback. We address each comment below and have incorporated the corresponding revisions into the manuscript, highlighted in pink.

W1. Text quality is mostly judged by perplexity scores. However, it only measures fluency. For reliable evidence, the authors should have checked the text quality on some downstream tasks like summarization.

While PPL is an important measure of fluency, it may not fully capture linguistic degradation. To complement PPL, we also use GPT-5 as an LLM judge to assess grammatical correctness, coherence, readability, lexical appropriateness, and repetition (Appendix J). Figures 3 and 7 show that KGW-NT achieves a better detection–quality trade-off than the baselines under both metrics.

To further evaluate downstream utility, we evaluate French summarization on 500 OrangeSum examples using Llama-3.1-8B. For a fair comparison, we select configurations from each method's detection–quality Pareto frontier with comparable detection strength and use GPT-5 to assess factual fidelity, coverage, and linguistic quality (Appendix K). We also include the open-ended generation judge scores from Figure 10 for comparison.

The table below (Table 5 in the revised manuscript) shows that KGW-NT achieves the highest worst-case detection z-score, lowest PPL ratio, and highest judge scores among fingerprinted models. Notably, its summarization score remains close to the base model (7.15 vs. 7.21), demonstrating that KGW-NT preserves quality in downstream summarization as well as open-ended generation. The smaller quality degradation observed in summarization across methods may reflect its relatively low-entropy generation, as outputs are constrained by the source text and often reuse its vocabulary and phrasing. This is consistent with prior findings that watermark detectability is stronger in high-entropy settings [1]. These results suggest that summarization alone may not fully capture watermark-induced quality degradation in open-ended generation.

MethodWorst-z ↑Summ. Judge ↑Open-ended Judge ↑PPL Ratio ↓
Base—7.217.691.00
KGW8.057.046.392.16
SWEET8.447.106.791.60
MorphMark7.746.876.461.88
KGW-TK9.476.396.341.79
KGW-NT (Ours)10.057.157.601.07

W2. The perplexity is calculated by Qwen-2.5-32B. Given that the targeted LLMs include Qwen-2.5 variants, this setting is biased and unfair to Llama-3 models.

Q1. What is the reason for using Qwen2.5 rather than Qwen3.x?

For a fair comparison with [2], whose utility evaluation we revisit, we follow the same experimental setup as [2], including the use of Qwen2.5-32B as the PPL reference model and the same target models (Llama-3.2-3B, Qwen2.5-3B, and Llama-3.1-8B).

To address potential evaluator-family bias, we additionally evaluate PPL using Gemma-2-27B as an independent reference model. Figure 14 shows consistent detection–quality trends with those obtained using Qwen2.5-32B, with KGW-NT maintaining its advantage over the baselines.

W3. The proposed method requires tuning the hyperparameter. Given that tuning the threshold of epsilon sampling can improve text generation performance in natural language generation fields, readers cannot judge whether text quality is improved by the proposed method itself or by improved sampling through careful hyperparameter tuning.

Q2. How did you decide on each hyperparameter?

KGW-NT does not use epsilon sampling or modify the inference-time decoding strategy of the fingerprinted model. Instead, its threshold τ\tau determines which green tokens receive the watermark bias during teacher generation.

Importantly, the near-tie threshold τ\tau can be calibrated directly from training data without extensive hyperparameter tuning. Specifically, we evaluate thresholds corresponding to the 10th, 30th, 50th, and 70th percentiles of the training-data logit-gap distribution. The 30th–50th percentiles consistently yield favorable detection–quality trade-offs across models (Figure 5 and Tables 7-8), suggesting that KGW-NT does not require careful model-specific threshold tuning.

Furthermore, for a fair comparison, we systematically sweep the hyperparameters of all methods and compare their detection–quality Pareto frontiers (Figure 3). We evaluate 12 configurations each for KGW-NT, SWEET, MorphMark, and KGW-TK, and 3 for KGW, which has only the logit-bias strength as a tunable hyperparameter (Appendix E).

[1] Lee, et al. Who wrote this code? watermarking for code generation. ACL 2024

[2] Gloaguen, et al. LLM fingerprinting via semantically conditioned watermarks. ICLR 2026.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you again for your review. We have provided clarifications and additional experimental results to address your concerns. We would greatly appreciate any further feedback you may have and would be happy to provide additional clarifications if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.