- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose DGMark, a training-free, attention-guided watermarking framework for Large Vision-Language Models (LVLMs) operating on frozen model weights. The method targets two specific failure modes in prior entropy-adaptive LVLM watermarking (specifically AGMark): (1) partition collapse, where scaling the semantic-critical token set size by a single normalized Shannon entropy leads to a rounded set size of zero when the modulated ratio is small, causing degradation to random partitioning; and (2) low-entropy distortion, where additive green-list logit biasing at high-confidence decoding steps overrides the model's sharp top-1 preference.
Critical strengths
The identification of integer rounding collapse in adaptive partitioning and distribution distortion in peaked distributions targets two genuine, overlooked edge cases in entropy-adaptive LVLM watermarking. Pairing multi-aspect distribution metrics (macro Shannon entropy + micro margin + global Gini) with a post-rounding floor is mathematically clean, intuitive, and preserves continuous modulation across the rest of the spectrum.
Critical weaknesses
-
There are multiple glaring contradictions across the manuscript regarding the primary evaluation metrics:
- Z-score contradiction: In Section 4.2, the text states: "The Z-score of 5.12 confirms strong watermark signal strength with reliable statistical detectability." However, the Abstract, Figure 1, and Table 2 all report Z-scores ranging from 10.95 to 11.02.
- Perplexity degradation () contradiction: In Section 4.2 , the authors report: "The relative perplexity increase is only (14.3% increase from plain text)". In stark contrast, Section 4.4, Table 4, and the Conclusion claim negative degradation ( and ), highlighting this as a major breakthrough. These discrepancies undermine the reliability of the empirical findings.
-
Methodological Deficiencies in Text Quality and Baseline Comparisons:
- In Table 1, baseline perplexities are directly copied from prior literature, while DGMark reports 23.78, explained by the use of an internal GPT-2-large model. Because baselines were not re-evaluated under this exact GPT-2-large pipeline, direct comparison of fluency degradation between DGMark and baselines is impossible.
- The evaluation is carried out on only 50 samples on a single benchmark (AMBER) using a single model family (Qwen3-VL-8B-Instruct). For claims of 100.00% AUC and 99.00% accuracy, a test set of 50 samples provides an insufficient sample size to guarantee statistical power or generalize across diverse multimodal domains.
Questions for authors
- Resolution of Numerical Inconsistencies:*
- Which Z-score is correct: 5.12 (reported in Section 4.2) or 11.02 (reported in the Abstract, Figure 1, and Table 2)?
- Which perplexity degradation figure is valid: (reported in Section 4.2) or (reported in Table 4 and Section 4.4)? Please clearly explain what configuration produced each number.
- Can you provide the perplexity (and ) of the baseline methods (especially AGMark and MorphMark) evaluated under the exact same local GPT-2 setup used for DGMark?
- Calculating the Gini coefficient over the complete vocabulary () requires sorting all logits (). Appendix C.1 claims this adds only 0.8ms per step. Is this calculation computed over the full 151,936 vocabulary or truncated to a top- / top- subset? If computed over the entire vocabulary, how is sub-millisecond execution achieved on standard hardware?
Response to Reviewer 8fq2 — Configuration Mapping, Fair PPL Comparison, and Efficiency Clarification
Dear Reviewer,
Thank you for your careful review and constructive comments. We are pleased that you recognized rounding collapse and peaked-distribution distortion as genuine edge cases and our multi-aspect design as clean and intuitive. Regarding numerical consistency, fair comparison, and efficiency, we respond below.
Response. Both sets are correct but from different configurations, and our presentation failed to map them. Z=5.12 and ΔPPL =+2.98 are the Table 1 main DGMark setting vs plain text. Z=10.95-11.02 and ΔPPL =−0.22/−0.18 are the VTDG-Fusion αvt sweep in Appendix, with best quality at αvt=1.0/0.5. VTDG-Fusion varies αvt∈{0,0.3,0.5,0.7,1.0} with st=βvt+(1−β)ct, β=0.7; detection stays 10.95-11.02 with green ratio >0.98 while BERTScore rises 97.56->97.88. We will add a configuration-mapping table stating model, benchmark split, n, αvt, length, and which Z/ΔPPL belongs to which table, and separate Main Z from Fusion Z in Abstract/Sec.4.
Response. We agree absolute PPL is not comparable across papers and already state this; our SOTA claim rests on ΔPPL plus BERTScore/STS, not PPL 23.78 itself. We use an external smaller LM to avoid self-scoring bias. We will re-evaluate AGMark/MorphMark under the exact same local GPT-2 setup and report PPL$_{wm}$ and ΔPPL side by side, plus mean lengths, length-normalized Z, and CIs. We use n=50/config on AMBER following AGMark/VLA-Mark LVLM protocol due to LVLM cost; the method is training-free on frozen logits/attention. We will add seeds/variance and state single-model/single-benchmark as a limitation.
Response. ∣V∣=151936. The 0.8ms is measured mean per step on A100 for vectorized margin plus Gini plus fusion, within 45ms->47.1ms total (4.7%); LES triggers on 31.4% steps with 87.6% first-try success. We will clarify whether Gini uses full logits or top-k/top-p candidates, give full profiling breakdown with batch/hardware/repeats, and correct Appendix C.1 if truncated computation was used.
We hope these clarifications address your concerns. If they do, we would be grateful if you could consider updating your score. We welcome further discussion. Thank you again.