- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper combines SFT and IoU-rewarded GRPO for bounding-box generation with inference-time interpolation between a VLM's next-token distribution and tokens supported by predefined Visual Grounding Graphs. It reports improved entity matching with limited decoding overhead. The problem is relevant, but the submission does not sufficiently establish novelty or hallucination reduction. Unspecified VGG provenance and inconsistencies between the equations and reported quantities substantially limit confidence in the conclusions.
Critical strengths
- Object-specific grounding addresses a relevant source of errors in visual descriptions.
- The interpolation parameter is interpretable, and the evaluation considers localization, fluency, latency, and capability preservation.
Critical weaknesses
-
The contribution of graph topology is unclear. Equations (3), (8), and (9) do not use graph edges; a box-to-attribute lookup table appears equivalent. Interpolation and IoU-rewarded GRPO have precedents in kNN-LM (Khandelwal et al., 2020) and Visual-RFT (Liu et al., 2025). Page 2 replaces PageRank with IoU localization, yet page 9 invokes undefined PageRank convergence and self-attention modulation. The incremental contribution needs clarification.
-
VGG provenance and the deployment setting are unspecified. The source of inference-time boxes and attributes is unclear. Evaluation annotations imply an oracle setting; automatic prediction requires construction accuracy, error propagation, and total latency. Baselines need equivalent information. Appendix B checks training-text insertion only, leaving graph reliability and deployment without annotations unestablished.
-
Reported quantities conflict with the mathematical definitions; implementation details and logs are needed.
-
Mean advantage. Equation (6) defines , giving for complete, equally weighted groups with nonzero variance. Section 4.1 and Table 1 instead report rising from 0.10 to 0.78.
-
GRPO loss. Equation (7)'s clipped term is bounded above by . Since , complete-group averaging and nonnegative KL imply . Table 1 reports -0.260 at step 400 and -0.755 at step 2,000. Different scaling or aggregation must be specified.
-
Probability normalization. Figure 1(b)'s values (0.680, 0.640, 0.140, 0.120) sum to 1.58. Yellow and green imply ; if red and ripe are the two supported tokens, normalized fusion gives 0.38 and 0.34. The displayed values match an unnormalized gate.
-
Factuality guarantees. Unsupported tokens can retain positive for . Allocating 60% of mass across supported tokens does not guarantee they win greedy selection. Section 3.3 overstates the guarantees.
-
-
Hallucination evaluation and model coverage are insufficient. Equation (11) measures coverage without explicitly penalizing hallucinations, while Eq. (10) supplies target region descriptions. POPE comparisons are limited and do not clearly evaluate the complete method. CHAIR, AMBER, MMHal-Bench, and HallusionBench results are absent, and only Qwen2-VL-7B-Instruct is tested.
-
Additional experimental details require clarification. Fluency intervals (±0.02 to ±0.04, Tables 5–6) exceed the normal-approximation 95% half-width bound of 0.0035 if all 1.26M queries are averaged; clarify the bootstrap procedure. The RefCOCO testA baseline (15.8%) versus Wang et al. (2024)'s 93.6% requires protocol and parsing clarification. Training datasets differ between Section 3.1 and Appendix B.2; specify the RefCOCO/+/g splits and deduplication.
Khandelwal et al. (2020). Generalization through Memorization: Nearest Neighbor Language Models. ICLR.
Liu et al. (2025). Visual-RFT: Visual Reinforcement Fine-Tuning. arXiv:2503.01785.
Wang et al. (2024). Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. arXiv:2409.12191.
Questions for authors
- Can the VGG construction procedure, annotation access, and information available to each baseline be specified?
- Can the metric/loss calculation code and training logs reconcile the advantage, loss, and probability-normalization discrepancies above?
- Can the complete decoder be evaluated with direct hallucination metrics, matched-information ablations, and an additional backbone?
The authors did not respond during the discussion period, so my concerns about how the VGGs are built at inference time, the mismatch between Eq. 9 and Fig. 1(b), and the faulty references (including the one cited for the core SFT and GRPO method) remain, and I am lowering my rating to 1 (clear rejection)