- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes an improvement over watermark-based fingerprinting method. They show that on the fingerprint domain, the watermark significantly degrades the generated text quality and that lowering the watermarking strength can improve quality but decreases the fingerprint detectability. They find that, in practice, the fingerprint does not need to use the full watermark, and that restricting the watermark to only the most likely tokens keep a strong fingerprinting signal without significantly impacting quality.
Critical strengths
- The idea of using a "weaker" watermark teacher for fingerpriting is intuitive, and the proposed near-tie teacher appears to be very effective at preserving the fingerprint detectability while limiting the impact on model quality.
- The paper is very easy to follow and the evaluation is thorough.
Critical weaknesses
- The novelty of this work is quite limited. This work iteratively improves the method established in [1], and the implication "aggregating signals across queries make sparser signals viable" seems to be similar to the idea in [1] of reducing the watermark strength . The framing of this work is also a bit aggressive. Even though the authors argue the evaluation protocol in [1] favors overly strong teacher, it seems the key issue is a mislabelling of a given metric (which should be reported to the authors of [1]) and that the "normal" watermark distillation induces too high distortion.
- The near-tie watermark intuitively seems to have a similar effect as lowering the temperature (because it increases the student probability of sampling a high probability teacher token) which mechanistically increases the PPL. In Table 4, we see that, for some models, the PPL of the fingerprinted KGW-NT model is lower than the base model, but (assuming the base model has the "best" quality) that suggests still a quality degradation. A better metric might be the L1 norm of the base model PPL and the fingerprinted model PPL.
[1] LLM fingerprinting via semantically conditioned watermarks, Gloaguen et al., ICLR 2026
Questions for authors
- As the authors mentioned, a watermark-based fingerprint only needs a sparse signal to be detected. The proposed method still embeds the watermark in every token (eventhough restricted to the top logits). Have the authors also explored more deviation from the KGW watermarks? I believe exploring to what extend the watermark needs to be embedded is an important direction.
- Could the authors show that the lower PPL translates in low quality impact and is not only an artifact of decreasing temperature? Maybe metrics like pass@k on hard benchmarks could distinguish such cases?
4 comments
Official Comment by Authors (2/2)
W2. The near-tie watermark intuitively seems to have a similar effect as lowering the temperature (because it increases the student probability of sampling a high probability teacher token) which mechanistically increases the PPL. In Table 4, we see that, for some models, the PPL of the fingerprinted KGW-NT model is lower than the base model, but (assuming the base model has the "best" quality) that suggests still a quality degradation. A better metric might be the L1 norm of the base model PPL and the fingerprinted model PPL.
Q2. Could the authors show that the lower PPL translates in low quality impact and is not only an artifact of decreasing temperature? Maybe metrics like pass@k on hard benchmarks could distinguish such cases?
To examine whether temperature-based sharpening can provide the gains of KGW-NT, we extend our original temperature analysis on Llama-3.1-8B to all baselines and additionally evaluate Qwen2.5-3B, sweeping the decoding temperature while keeping KGW-NT fixed at . The revised Figure 13 shows that KGW-NT maintains a favorable detection–quality Pareto frontier even when the baselines are allowed to tune their decoding temperatures. These results indicate that temperature tuning alone does not provide KGW-NT's gains in this setting.
Regarding PPL ratios below 1, the base model's PPL should not be treated as the optimal target. Since PPL is measured using an external reference model, a PPL ratio below 1 does not imply quality degradation. Likewise, the absolute PPL difference is not necessarily a better quality metric, as matching the base model's PPL does not guarantee better text quality.
To complement PPL, we also evaluate linguistic quality using an LLM judge that assesses grammatical correctness, fluency, coherence, readability, lexical appropriateness, and repetition (judge prompt provided in Appendix J). Figures 3 (right) and 7 (bottom) show no apparent quality degradation relative to the base models in settings where PPL ratios fall below 1.
To further assess these settings, we evaluate mathematical reasoning on 250 French MGSM problems using Qwen2.5-3B, with 16 samples per problem at and top-. The table below (also Table 6 in the revised manuscript) shows that KGW-NT maintains pass@k performance comparable to the base model despite its lower PPL. In particular, KGW-NT() achieves 73.1% pass@1 and 96.8% pass@16, compared with 71.5% and 97.2% for the base model, while also achieving a higher LLM judge score (7.66 vs. 7.27). These results provide further evidence that lower PPL does not necessarily come at the expense of linguistic quality or reasoning performance. We also confirm that KGW-NT preserves downstream utility on the summarization task (Appendix I.8 and Table 5).
| Method | PPL ratio ↓ | Worst-z ↑ | Judge ↑ | Pass@1 ↑ | Pass@4 ↑ | Pass@16 ↑ |
|---|---|---|---|---|---|---|
| Base | 1.00 | — | 7.27 | 71.50 | 90.20 | 97.20 |
| KGW () | 1.27 | 8.66 | 6.81 | 68.30 | 89.20 | 96.80 |
| SWEET | 1.02 | 6.37 | 7.32 | 70.60 | 89.60 | 96.00 |
| MorphMark | 1.02 | 7.30 | 7.21 | 71.00 | 89.30 | 95.20 |
| KGW-TK | 0.87 | 7.52 | 7.26 | 69.80 | 89.30 | 95.60 |
| KGW-NT () | 0.78 | 6.66 | 7.56 | 72.30 | 90.20 | 96.80 |
| KGW-NT () | 0.79 | 9.31 | 7.66 | 73.10 | 90.60 | 96.80 |
Thank you for your reply and additional experiments, however I still have some remaining concerns.
- I believe I have not misunderstood your key contributions and methodology. My concern was about the depth of your investigation. In its current form, the method proposed is quite narrow and I think this work would benefit from deeper deviation from [1]'s method.
- Thanks for the additional experiment, it confirms that the method is effective at fingerprinting. My concerns in that regard have been addressed. Overall, the authors rebuttal experiments confirm my review: the proposed method is sound and indeed improves the fingerprinting performance (better quality-detectability trade-off) compare to [1]. However, I still find the investigation limited with little informativeness. Thus, I will keep my score.
Official Comment by Authors
Thank you for acknowledging that our method is sound, improves the detection–quality trade-off over [1], and that our additional experiments have addressed your concerns regarding its effectiveness.
While you state that you have not misunderstood our key contributions or methodology, our concern about potential misunderstandings was based on specific factual inaccuracies and mischaracterizations in the original review. We addressed these points in our earlier rebuttal and briefly reiterate them below.
Q1. "The proposed method still embeds the watermark in every token"
This is factually incorrect. KGW-NT applies no watermark bias at positions where no green token satisfies the near-tie condition (Section 5.3).
W1. "aggregating signals across queries make sparser signals viable" seems to be similar to the idea in [1] of reducing the watermark strength .
Simply weakening the watermark strength does not resolve the detection–quality trade-off: Figure 2 shows that still more than doubles PPL, whereas falls below the detection threshold. Instead, guided by Proposition 5.1, we selectively restrict which tokens receive watermark bias rather than reducing the bias strength .
Q1. "Have the authors also explored more deviation from the KGW watermarks?"
We have already extended the near-tie restriction to SWEET and MorphMark, demonstrating improved detection–quality trade-offs (Figure 6). Compatibility with existing watermarking schemes is an important strength, particularly given the diversity of advanced watermarking methods already available.
W1. "it seems the key issue is a mislabelling of a given metric"
Our contribution goes beyond correcting a mislabeled metric. We uncover substantial open-ended generation quality degradation overlooked by [1]'s evaluation protocol (Table 1), revealing that the prior evaluation favors overly strong watermark teachers.
We therefore respectfully disagree with your assessment that our investigation is limited and offers little new insight. Beyond the proposed method, we identify overlooked quality degradation in existing fingerprinting evaluations and provide theoretical and empirical insights into sparse watermark signal placement. Our broader finding is that watermark teachers should be designed specifically for model fingerprinting, and our analysis provides concrete guidance for doing so. We believe our contributions should be assessed based on the new findings and insights they provide, rather than solely on the degree of methodological deviation from [1].
Official Comment by Authors (1/2)
We appreciate the reviewer's recognition of our method's effectiveness and the clarity of our presentation. However, we are concerned that some of the comments may stem from misunderstandings of our key contributions and methodology. We clarify these points below and have incorporated the corresponding revisions into the manuscript, highlighted in pink.
Our contribution goes beyond correcting a mislabeled metric or simply weakening the KGW watermark teacher. While [1] establishes an effective fingerprinting framework, its utility evaluation primarily relies on accuracy-based benchmarks and reports mean token entropy as PPL, overlooking substantial quality degradation in open-ended generation (Table 1). Importantly, even under its multi-query verification protocol, reducing the KGW bias strength does not resolve the detection–quality trade-off: Figure 2 shows that δ=2 still more than doubles PPL, whereas δ=1 falls below the detection threshold.
To overcome this trade-off, we reconsider whether conventional text watermark teachers are well suited for model fingerprinting. Unlike text watermarking, which typically requires detection from individual outputs, fingerprint verification aggregates watermark signal across queries, allowing sparser signals. Yet [1] retains the conventional KGW teacher that biases every green token, without exploiting this distinction in teacher design.
This leads us to investigate where to place the sparse watermark signal, rather than merely how much to weaken it. Our token-surprisal analysis (Proposition 5.1) shows that, even at matched watermark strength, biasing more plausible tokens induces a smaller surprisal shift. Guided by this insight, we introduce near-tie restriction, which biases only green tokens close to the top-1 logit. Unlike conventional KGW, it does not apply bias at every token: if no green token satisfies the near-tie condition, no bias is applied (Section 5.3). Since near-tie restriction simply limits the set of biased green tokens, it can also be combined with other watermarking schemes.
Empirically, KGW-NT achieves a better detection–quality trade-off than the baselines, including KGW-TK, which restricts the bias to top-k tokens (Figure 3). This highlights the importance of logit-gap-based signal placement beyond simply restricting the biased token set. We further examine the degree of restriction by varying τ across the 10th, 30th, 50th, and 70th percentiles of training-data logit gaps, finding favorable detection–quality trade-offs at the 30th–50th percentiles (Figure 5). Furthermore, applying near-tie restriction to SWEET and MorphMark also improves their detection–quality trade-offs (Figure 6), demonstrating its compatibility across different watermarking schemes.