- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper is working on VLM object hallucination issue, and propose a training-free decoding method FaiD. Specifically, it proposes to perturb top-k visual tokens and measure logits change to get sensitivity score. Leveraging this score, FaiD the combines a AGLA-derived positive visual view with a correction based on original-versus-perturbed logits. Experiment over three backbones and four benchmarks demonstrate certain benefits.
Critical strengths
-
The paper is well presented and crafted method is simple and easy to use.
-
The reported improvements span several model families, with useful ablations. The paper examines alternative control signals, perturbation strategies, single-branch corrections, gating functions, and generation budgets.
Critical weaknesses
-
The proposed method and analysis has little innovation. VGS-Decoding [1] already uses sensitivity to visual corruption to adapt candidate-token probabilities. CHASD [2] uses attention-selected regions for localized perturbation and conditional contrastive correction. Neither is discussed. FaiD’s continuous combination with a positive visual view may be distinct, but the evidence for its particular signal is limited.
-
Propose method will induce substantial computational overhead. For example, on LLaVA1.5, FaiD takes18.6 seconds versus 9.4 seconds for AGLA, while improving CHAIR-S from 49.2% to 44.8%. This makes the additional inference cost consequential for deployment.
-
There are missing of multiple recent baselines in comparison: VGS-Decoding[1], CHASD[2], SID[3]
[1] https://arxiv.org/abs/2603.20314 [2] https://arxiv.org/abs/2605.23344 [3] https://proceedings.iclr.cc/paper_files/paper/2025/file/3cc87f2bd3e3b4df8f9217326761c322-Paper-Conference.pdf
Questions for authors
- How are pseudo-grounded and pseudo-hallucinated tokens labeled? Which data are used for calibration and hyper-parameter selection, and are they separate from evaluation? Please also specify the Gaussian noise scale and sampling settings.
Response to Reviewer RRSo
We sincerely thank you for recognizing the clarity and simplicity of FaiD and its improvements across multiple LVLMs. We appreciate the concerns about innovation, computational overhead, and the coverage of recent decoding baselines.
Response: Thank you for highlighting these closely related methods. FaiD differs from these methods in what the adaptive signal controls. It uses candidate-specific responses to visual intervention to continuously allocate weight between positive enhancement and contrastive correction, rather than adapting only negative-view construction, candidate probabilities, or branch activation.
We additionally compare CHAIR results on LLaVA-1.5-7B. SID is reproduced using its official implementation under matched settings. We omit a numerical comparison with VGS-Decoding because its reported evaluation concerns medical VQA rather than our general-domain setting.
Table R11. Comparison of multiple recent baselines.
In the comparison, FaiD lowers CHAIR-S by 3.4% relative to SID, while SID achieves lower CHAIR-I. Their recall and response lengths are similar, indicating different sentence-level and instance-level hallucination trade-offs. We have added the methodological discussion to the Introduction and Related Work, and the result comparison to Table 1.
Response: We acknowledge that FaiD introduces additional inference overhead due to visual support probing and positive evidence enhancement. However, the overhead varies across LVLMs. As shown in Table 10, on Qwen2.5-VL, FaiD reduces CHAIR-S from 27.4% to 19.8% compared with VCD, with latency increasing from 28.7s to 31.9s. Similarly, on InternVL3, FaiD reduces CHAIR-S from 31.4% to 23.6%, while latency increases only from 20.5s to 21.2s. Of course, we agree that reducing inference overhead remains an important direction for future improvement.
Response: We acknowledge the insufficient implementation details. We clarify the labeling procedure, parameter selection, noise scale, and decoding settings below.
Labels. We match generated object mentions to the CHAIR vocabulary and synonym mapping. A mention is pseudo-grounded if its normalized object category occurs in the corresponding MSCOCO-derived CHAIR annotations, and pseudo-hallucinated otherwise.
Parameter selection. Consistent with the hyperparameter selection protocol in prior training-free decoding studies, such as VCD[1], AGLA[2], and DeGF[3], we evaluate the faithfulness margin m and perturbation ratio K on CHAIR. We did not construct a separate held-out calibration split.
[1] Sicong Leng, et.al. Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding. In CVPR, pp. 13872–13882, 2024.
[2] Wenbin An, et.al. Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention. In CVPR, pp. 29915–29926, 2025.
[3] Ce Zhang, et.al. Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models. In ICLR, 2025.
Gaussian noise scale and sampling setting. We define the perturbation noise as standard Gaussian, ϵ∼N(0,I), with zero mean and unit standard deviation (σ=1). We use greedy next-token selection, with a maximum of 1,024 new tokens for generative evaluation.