- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SIEVE, a training-free framework for mitigating object hallucinations in large vision-language models. Unlike approaches relying primarily on the generator's internal information, SIEVE uses an external object detector to verify generated object claims against visual evidence. When a claim is rejected, generation rewinds to the beginning of the rejected word, blocks the corresponding token at that position, and resumes decoding. The regenerated claims are verified iteratively until accepted. The authors further introduce a Rescue–Damage criterion for selecting the external verifier and employ Learn-then-Test (LTT) to calibrate the verification threshold. Experiments across three LVLMs demonstrate substantial reductions in object hallucinations.
Critical strengths
1. Simple and effective framework. SIEVE combines external verification with iterative rewind decoding without requiring additional training. The approach is straightforward and achieves promising results across different LVLMs.
2. Good experimental performance. The method substantially reduces object hallucinations while maintaining reasonable decoding costs. The comparisons and ablation studies provide useful evidence for its effectiveness.
Critical weaknesses
1. Limited novelty and insufficient validation of iterative verification. The main approach combines existing ideas, including external visual verification and iterative correction. Although the combination is interesting, its novelty appears somewhat limited. Moreover, Table 6 examines different post-rejection actions but does not isolate the benefit of iterative verification itself. A direct comparison with one-shot external verification under the same evidence provider and decoding settings would better establish the contribution of the proposed mechanism.
2. Strong dependence on the external Evidence-provider. The effectiveness of SIEVE depends heavily on the accuracy and category coverage of the external object detector. For example, Table 4 reports a Rescue rate of 0.50 and a Damage rate of 0.17 for Faster R-CNN, indicating that some unsupported claims may remain undetected while valid claims may be incorrectly rejected. The robustness of the framework to verifier errors and unsupported object categories is not sufficiently investigated.
Questions for authors
1. Limitations of the external evidence provider. How does SIEVE handle object categories outside the detector's supported vocabulary? Could verifier errors or unsupported categories lead to additional hallucinations during regeneration? A more detailed analysis of these cases would be helpful.
2. Contribution of iterative verification. Could the authors provide a direct comparison between one-shot external verification and iterative rewind decoding using the same evidence provider and decoding settings? This would help clarify the independent contribution of iterative correction.
3. Repeated rejections and termination. What happens when multiple consecutive candidate objects are rejected? Is there a maximum number of rewinds or a fallback mechanism to ensure termination?
4. LTT calibration and hyperparameter selection. Could the authors clarify the calibration sample size, statistical testing procedure, confidence parameter, and candidate threshold selection used in LTT? A sensitivity analysis of the Damage tolerance and calibration settings would also help assess the robustness of the method.
5. The manuscript appears to omit the AI Use Statement and others required by the ICLR 2027 submission guidelines.