acceptodds
Under review as a conference paper at ICLR 2027

DGMark: Dual-Granularity Composite Entropy for Robust Vocabulary Partitioning in Vision-Language Model Watermarking

Abstract

Watermarking large vision-language models embeds a detectable signal into generated text by dynamically partitioning the vocabulary into green and red lists at each decoding step, using a single entropy value to size the partition. This design has two shortcomings. The entropy-modulated ratio is rounded to an integer list size, and when the ratio is small relative to the vocabulary, the rounded size collapses to zero, silently discarding the semantic-critical subset and degrading the mechanism to a random partition; separately, at genuinely low-entropy steps, additively biasing green-list logits overrides the model's already-concentrated top-1 preference, distorting the output distribution precisely where the model is most confident. We propose two lightweight, training-free mechanisms that address these gaps while keeping the underlying LVLM frozen. A dual-granularity composite entropy signal fuses normalized Shannon entropy with the top-1/top-2 probability margin and the Gini coefficient, paired with a list-size floor that provably keeps the semantic-critical subset non-empty at every step without disturbing the continuous modulation elsewhere. A low-entropy rejection sampling scheme replaces additive logit biasing with resampling directly from the original distribution at low-entropy steps, preserving natural confidence structure while still enforcing a detectable signal. We evaluate on Qwen3-VL-8B-Instruct using the AMBER benchmark, reporting perplexity, BLEU, and BERTScore alongside entropy-bucketed statistics of the adaptive green-list ratio. Together, these two mechanisms fix the partition-collapse and low-entropy distortion failure modes in attention-guided LVLM watermarking, improving semantic fidelity without sacrificing detectability or efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Vision-Language Models, Watermarking, Vocabulary Partitioning, Composite Entropy, Attention Guidance, Detection Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_8fq2OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose DGMark, a training-free, attention-guided watermarking framework for Large Vision-Language Models (LVLMs) operating on frozen model weights. The method targets two specific failure modes in prior entropy-adaptive LVLM watermarking (specifically AGMark): (1) partition collapse, where scaling the semantic-critical token set size by a single normalized Shannon entropy leads to a rounded set size of zero when the modulated ratio is small, causing degradation to random partitioning; and (2) low-entropy distortion, where additive green-list logit biasing at high-confidence decoding steps overrides the model's sharp top-1 preference.

Critical strengths

The identification of integer rounding collapse in adaptive partitioning and distribution distortion in peaked distributions targets two genuine, overlooked edge cases in entropy-adaptive LVLM watermarking. Pairing multi-aspect distribution metrics (macro Shannon entropy + micro margin + global Gini) with a post-rounding floor is mathematically clean, intuitive, and preserves continuous modulation across the rest of the spectrum.

Critical weaknesses
  1. There are multiple glaring contradictions across the manuscript regarding the primary evaluation metrics:

    • Z-score contradiction: In Section 4.2, the text states: "The Z-score of 5.12 confirms strong watermark signal strength with reliable statistical detectability." However, the Abstract, Figure 1, and Table 2 all report Z-scores ranging from 10.95 to 11.02.
    • Perplexity degradation (ΔPPL\Delta\text{PPL}) contradiction: In Section 4.2 , the authors report: "The relative perplexity increase is only ΔPPL=+2.98\Delta\text{PPL} = +2.98 (14.3% increase from plain text)". In stark contrast, Section 4.4, Table 4, and the Conclusion claim negative degradation (ΔPPL=−0.22\Delta\text{PPL} = -0.22 and −0.18-0.18), highlighting this as a major breakthrough. These discrepancies undermine the reliability of the empirical findings.
  2. Methodological Deficiencies in Text Quality and Baseline Comparisons:

    • In Table 1, baseline perplexities are directly copied from prior literature, while DGMark reports 23.78, explained by the use of an internal GPT-2-large model. Because baselines were not re-evaluated under this exact GPT-2-large pipeline, direct comparison of fluency degradation between DGMark and baselines is impossible.
    • The evaluation is carried out on only 50 samples on a single benchmark (AMBER) using a single model family (Qwen3-VL-8B-Instruct). For claims of 100.00% AUC and 99.00% accuracy, a test set of 50 samples provides an insufficient sample size to guarantee statistical power or generalize across diverse multimodal domains.
Questions for authors
  1. Resolution of Numerical Inconsistencies:*
    • Which Z-score is correct: 5.12 (reported in Section 4.2) or 11.02 (reported in the Abstract, Figure 1, and Table 2)?
    • Which perplexity degradation figure is valid: ΔPPL=+2.98\Delta\text{PPL} = +2.98 (reported in Section 4.2) or ΔPPL=−0.22\Delta\text{PPL} = -0.22 (reported in Table 4 and Section 4.4)? Please clearly explain what configuration produced each number.
  2. Can you provide the perplexity (and ΔPPL\Delta\text{PPL}) of the baseline methods (especially AGMark and MorphMark) evaluated under the exact same local GPT-2 setup used for DGMark?
  3. Calculating the Gini coefficient over the complete vocabulary (∣V∣=151,936|V| = 151,936) requires sorting all logits (O(∣V∣log⁡∣V∣)O(|V| \log |V|)). Appendix C.1 claims this adds only 0.8ms per step. Is this calculation computed over the full 151,936 vocabulary or truncated to a top-kk / top-pp subset? If computed over the entire vocabulary, how is sub-millisecond execution achieved on standard hardware?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 8fq2 — Configuration Mapping, Fair PPL Comparison, and Efficiency Clarification

Dear Reviewer,

Thank you for your careful review and constructive comments. We are pleased that you recognized rounding collapse and peaked-distribution distortion as genuine edge cases and our multi-aspect design as clean and intuitive. Regarding numerical consistency, fair comparison, and efficiency, we respond below.

Q1: Z-score and perplexity values across configurations.

Response. Both sets are correct but from different configurations, and our presentation failed to map them. Z=5.12Z=5.12 and Δ\DeltaPPL =+2.98=+2.98 are the Table 1 main DGMark setting vs plain text. Z=10.95Z=10.95-11.0211.02 and Δ\DeltaPPL =−0.22=-0.22/−0.18-0.18 are the VTDG-Fusion αvt\alpha_{vt} sweep in Appendix, with best quality at αvt=1.0\alpha_{vt}=1.0/0.50.5. VTDG-Fusion varies αvt∈{0,0.3,0.5,0.7,1.0}\alpha_{vt}\in\{0,0.3,0.5,0.7,1.0\} with st=βvt+(1−β)cts_t=\beta v_t+(1-\beta)c_t, β=0.7\beta=0.7; detection stays 10.95-11.02 with green ratio >0.98>0.98 while BERTScore rises 97.56->97.88. We will add a configuration-mapping table stating model, benchmark split, nn, αvt\alpha_{vt}, length, and which ZZ/Δ\DeltaPPL belongs to which table, and separate Main ZZ from Fusion ZZ in Abstract/Sec.4.

Q2: Same-pipeline baseline perplexity and sample scope.

Response. We agree absolute PPL is not comparable across papers and already state this; our SOTA claim rests on Δ\DeltaPPL plus BERTScore/STS, not PPL 23.78 itself. We use an external smaller LM to avoid self-scoring bias. We will re-evaluate AGMark/MorphMark under the exact same local GPT-2 setup and report PPL$_{wm}$ and Δ\DeltaPPL side by side, plus mean lengths, length-normalized ZZ, and CIs. We use n=50n=50/config on AMBER following AGMark/VLA-Mark LVLM protocol due to LVLM cost; the method is training-free on frozen logits/attention. We will add seeds/variance and state single-model/single-benchmark as a limitation.

Q3: Gini cost over full vocabulary.

Response. ∣V∣=151936|V|=151936. The 0.8ms is measured mean per step on A100 for vectorized margin plus Gini plus fusion, within 45ms->47.1ms total (4.7%); LES triggers on 31.4% steps with 87.6% first-try success. We will clarify whether Gini uses full logits or top-kk/top-pp candidates, give full profiling breakdown with batch/hardware/repeats, and correct Appendix C.1 if truncated computation was used.

We hope these clarifications address your concerns. If they do, we would be grateful if you could consider updating your score. We welcome further discussion. Thank you again.

Reviewer_YBfYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces DGMark, a method that refines the "semantic key set" by integrating Shannon entropy, the probability gap between the top-1 and top-2 tokens, and the Gini coefficient, while enforcing a non-empty lower bound. In cases of low entropy, the method employs a limited-iteration rejection sampling strategy. The approach is training-free; evaluated on Qwen3-VL and the AMBER benchmark (using 50 samples per configuration), it reportedly achieves an AUC of 100%, an accuracy of 99%, and a BERTScore of 97.88.

Critical strengths
  1. Focused Problem Definition: The paper addresses specific issues—namely, set-rounding effects and biases in low-entropy scenarios—and explores the trade-off between watermark detection and generation quality, which is of practical significance.

  2. Lightweight Design: The method requires no retraining, making it easy to integrate with frozen (pre-trained) models. The authors also provide ablation studies for individual components and an analysis of visual alignment strength.

Critical weaknesses
  1. Insufficient Core Justification: The original AGMark method utilizes a ceil function and a (1−Hnorm)(1-H_{\mathrm{norm}}) term, which contradicts the motivation described in this paper. Furthermore, the use of rejection sampling—even with a fixed "green" token list—alters the output distribution; the rationale behind these "fixes" requires further clarification.

  2. Lack of Experimental Comparability: The original baseline uses 400 samples and greedy decoding, whereas this paper uses 50 samples and introduces sampling; additionally, the perplexity (PPL) evaluation metrics differ. The Z-score comparison also suffers from inconsistent sequence lengths (128 tokens vs. 77 tokens).

  3. Errors in Figures and Data: In Figure 1, the BERTScore is listed as 97.13%, which conflicts with the 97.88% mentioned in the figure caption. The figures, text, and data require a systematic verification, and the manuscript as a whole needs further refinement.

Questions for authors
  1. Please specify the implementation of AGMark used and the actual decoding rules applied. Additionally, provide a verifiable definition for the concept of "preserving the confidence structure."

  2. Does the performance improvement still hold when the sample, reference text, and output lengths are standardized? Furthermore, is the 88.61% AUC figure for paraphrase attacks derived from an independent evaluation?

1 comment
AuthorsOpenReview ↗

Response to Reviewer YBFY — Implementation, Fair Comparison, and Data Correction

Dear Reviewer,

Thank you for your careful review and constructive comments. We are pleased that you recognized our focused problem definition and lightweight training-free design with component ablations. Regarding core justification, comparability, and data accuracy, we respond below.

Q1: Partition-sizing rule and distribution preservation under rejection sampling.

Response. Our presentation may have obscured the implementation reference. We implemented AGMark attention-guided scoring st=βvt+(1−β)cts_t=\beta v_t+(1-\beta)c_t with β=0.7\beta=0.7, ρ=0.3\rho=0.3, green ratio 0.5, δ=1.5\delta=1.5, T=1.0T=1.0 as in Reproducibility Statement. Whether the baseline uses round or ceil, a tiny entropy factor either collapses MtM_t to 0 or clamps many steps to one fixed size; our post-rounding ∣Mt∣=max⁡(1,round(∣V∣γt))|M_t|=\max(1,{\rm round}(|V|\gamma_t)) removes 12.3% zero-size steps to 0% while only 3.1% hit floor vs 42.7% if flooring the continuous factor. For verifiable “preserving confidence structure” we mean top-1 retention at low-entropy steps plus Δ\DeltaPPL/BERTScore: without LES BERTScore 97.88->96.12 and Δ\DeltaPPL -0.22->+0.41, with 87.6% first-try success and 2.3 attempts average. LES draws x(j)∼ptx^{(j)}\sim p_t up to K=5K=5, otherwise falls back to ptδp_t^{\delta}, so sampling is from the original distribution. We will specify AGMark version, decoding rules, and this definition in Sec.3.

Q2: Sample size, decoding, length-normalized comparison and paraphrase result.

Response. We use n=50n=50/config following AGMark/VLA-Mark LVLM protocol due to LVLM cost; baselines are quoted under same AMBER setting and our gains use identical generation length within our suite. We agree ZZ grows with T\sqrt{T}, so 128 vs 77 tokens are not directly comparable. We will add fixed-length controls, per-token ZZ, mean lengths, and CIs. PPL uses external GPT-2 to avoid self-scoring bias; absolute PPL is not compared, our claim uses Δ\DeltaPPL plus BERTScore/STS (91.46%→97.88%91.46\%\to97.88\%, +6.42pp). The 88.61% AUC is from an independent robustness suite with word-level perturbations, translation and paraphrasing vs 99.89% unattacked, 11.28pp drop. We will release attack configs and lengths.

Q3: Figure-text inconsistency between 97.13% and 97.88%.

Response. We thank you for catching this. The verified result is 97.88% from quality_metrics.csv (0.978803), consistent with Table 1 and Sec.4. The Figure PDF retains an outdated 97.13% label while the caption was updated. We will replace all figure labels, recheck every number, remove the local path in Sec.4 to preserve anonymity, and clarify the Main Z=5.12Z=5.12 vs VTDG-Fusion Z=11.02Z=11.02 configurations.

We hope these clarifications address your concerns. If they do, we would be grateful if you could consider updating your score. We welcome further discussion. Thank you again.

Reviewer_xUxWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In general, watermarking methods embed detectable signals into generated text based on a separate vocabulary. However, this may cause two problems. One is discarding the semantically critical subset from the text due to rounding the list size to an integer. The second is distortion of the output distribution caused by adding biases to green-list logits. To solve these problems, the authors propose DGMark, which can eliminate rounding-induced collapse and make the sampled token fall into the green list as much as possible while keeping the original model output distribution. Experimental results confirm that DGMark can outperform baselines in detection while keeping the text quality.

Critical strengths
  • The targeted issue that the watermarking itself causes the collapse of the text is essential in this field.
  • Optimizing list size is quite helpful for any watermarking approach and can be a strong contribution to the community.
  • To capture finer-grained confidence, the authors introduce normalized Shannon entropy and Gini coefficient. These are natural for handling biases caused by the partition.
  • Evaluation results show the improvement in detectability by DGMark while keeping the text quality.
Critical weaknesses
  • Even though the approach targets LVLMs, the visual-specific aspect is not strongly considered in the proposed method.
  • Apart from targeting LVLMs, each approach is quite similar to previous works targeting only text-based LLMs. The authors clarify which parts or concepts are essentially novel.
  • The overview figures lack the important information that this work targets generated text, not images.
  • Only one model, Qwen3-VL-8B-Instruct, is evaluated. Thus, the evaluation results do not support the general conclusion.
  • Calculating perplexity by GPT-2 is inappropriate because you target a stronger LM, Qwen3-VL-8B-Instruct.
Questions for authors
  • How did you decide the hyperparameters? Since the proposed method has many hyperparameters, you need to report them to justify the performance improvement.
  • Could you explain the reason why you only target Qwen3-VL-8B-Instruct?
  • When using BERTScore, you can specify the encoder by yourself. Why did you use bert-base-uncased, which doesn't have a strong correlation to human judgment compared with other strong encoder models?
  • Results of BLEU are influenced by the generated token length. Could you also report it?
1 comment
AuthorsOpenReview ↗

Response to Reviewer xUwX — Clarifications on Vision Grounding, Novelty, and Evaluation Protocol

Dear Reviewer,

Thank you for your careful review and constructive comments. We are pleased that you recognized partition-collapse as central to this field, our fine-grained confidence design as helpful, and our detectability gains with preserved quality.

Q1: Vision-specific contribution and novelty beyond text-only LLM watermarking.

Response. The vision grounding is in placement not only figures. We build on attention-guided scoring st=βvt+(1−β)cts_t = \beta v_t + (1-\beta)c_t with β=0.7\beta=0.7, where vtv_t is cross-modal attention relevance. Our novelty is (i) Htc=λ1Hnorm+λ2(1−mt)+λ3(1−Gt)H^c_t=\lambda_1 H_{\rm norm}+\lambda_2(1-m_t)+\lambda_3(1-G_t) fusing Shannon, top-1/top-2 margin mtm_t and Gini GtG_t; (ii) post-rounding floor ∣Mt∣=max⁡(1,round(∣V∣γt))|M_t|=\max(1,{\rm round}(|V|\gamma_t)) eliminating 12.3% zero-size collapse to 0% while only 3.1% steps hit floor; (iii) first STA-style LES for attention-guided LVLM: at low-entropy steps sample x(j)∼ptx^{(j)}\sim p_t up to K=5K=5 until x∈Greenx\in Green. Without LES BERTScore 97.88->96.12 and Δ\DeltaPPL -0.22->+0.41. Appendix αvt\alpha_{vt} ablation shows Z-score 10.95-11.02 stable while BERTScore 97.56->97.88 improves with alignment, confirming visual guidance helps quality without hurting detection. We will sharpen this delta vs. AGMark/VLA-Mark/IE/MorphMark in Sec.3.

Q2: Single-model evaluation and hyperparameter justification.

Response. We agree single-model is a limitation and will state it explicitly. We use Qwen3-VL-8B-Instruct on AMBER with n=50n=50/config following AGMark/VLA-Mark protocol; the method is training-free on frozen logits/attention and model-agnostic. Hyperparameters are fully reported in Reproducibility Statement and ablated in Sec.4: λ2=0\lambda_2=0 increases Δ\DeltaPPL by 0.24, λ3=0\lambda_3=0 by 0.15; floor strategy comparison; LES on/off; αvt∈{0,0.3,0.5,0.7,1.0}\alpha_{vt}\in\{0,0.3,0.5,0.7,1.0\}. Weights (0.5,0.3,0.2)(0.5,0.3,0.2) were fixed before main comparison, not tuned per test. We will add a sensitivity note.

Q3: PPL evaluator, BERTScore encoder, and BLEU length effect.

Response. We use an external smaller LM for PPL precisely to avoid self-scoring bias where Qwen3-VL scores its own text favorably; this is common in watermarking. Absolute PPL is therefore not comparable across papers, which we already state. Our claim relies on Δ\DeltaPPL == PPL${wm}-PPLPPL{plain}$ plus model-agnostic BERTScore/STS: main result Δ\DeltaPPL =+2.98=+2.98 controlled, and αvt=1.0\alpha_{vt}=1.0 achieves Δ\DeltaPPL =−0.22=-0.22. BERTScore uses default bert-base-uncased identically for all methods, so relative gain 91.46%→97.88%91.46\%\to97.88\% (+6.42pp) is valid; we agree stronger encoders are future work. For BLEU we agree length matters; BLEU is secondary (31.13 vs AGMark 31.88) and our SOTA claim rests on BERTScore/STS. We will add mean generated length and length-normalized analysis in revision.

We hope these clarifications address your concerns. If they do, we would be grateful if you could consider updating your score. We welcome further discussion. Thank you again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.