- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Kestrel, a training-free framework for mitigating hallucinations in large vision-language models (LVLMs). It first converts an image question into verifiable claims, uses SAM3 to collect visual grounding evidence, and constructs structured textual observations from the grounding outputs. The target LVLM then verifies the claims and performs evidence-gated, history-aware refinement. Experiments with Qwen3-VL-8B and InternVL3.5-8B report improvements on POPE and MME-Hallucination. The paper also includes component ablations, prediction-transition statistics, efficiency measurements, and additional evaluations in the appendix.
Critical strengths
1. Relevant problem and interpretable design. The paper addresses residual hallucinations in stronger LVLMs and the risk of unnecessarily changing correct predictions. Using explicit visual evidence to guide verification and correction is well motivated, and the intermediate claims and evidence references make the decision process easier to inspect.
2. Useful empirical evaluation. The paper compares against a range of training-free mitigation methods on two recent LVLM backbones. It reports improvements across the main benchmarks and includes ablations, efficiency analysis, prediction transitions, and supplementary CHAIR and seed-robustness results.
Critical weaknesses
1. Limited methodological novelty. Kestrel largely integrates established ideas, including claim construction, external visual grounding, LVLM-based verification, and iterative correction. Woodpecker, for example, has already explored a multi-stage pipeline involving visual knowledge validation and hallucination correction. Kestrel differs in its SAM3-based evidence collection and conservative update strategy, but the paper does not make a sufficiently convincing case for the methodological significance of these additions beyond system integration.
2. Limited evidence for the contribution of the central refinement mechanisms. In Table 3, removing the evidence-gated update lowers POPE MS-COCO accuracy from 89.45% to 89.30%, while removing history-aware refinement yields 89.32%. These gains of 0.15 and 0.13 percentage points are small relative to the improvement of the complete system. Moreover, Appendix G reports up to 0.14 points of variation across random seeds on POPE MS-COCO. The ablations therefore do not yet convincingly establish the distinct benefits of the two mechanisms emphasized in the paper, although the overall evidence-based pipeline does improve performance.
3. Insufficient assessment of verification reliability. Existence and count evidence depend on SAM3 predictions, while some textual observations are produced by an LVLM; the target LVLM then performs verification and refinement. This design could propagate errors from grounding or evidence construction into subsequent decisions. The paper provides prediction-transition, qualitative, and seed-robustness analyses, but does not directly assess calibration of the verifier's confidence scores or how the gating mechanism responds to incorrect or incomplete external evidence.
4. Substantial inference cost relative to the incremental refinement gain. Appendix C (Table 6) reports average latency of 18.75 s for Kestrel versus 0.78 s for the Qwen3-VL baseline (24.03×). Accuracy after the first iteration is already 89.28%, compared with 89.45% for the complete pipeline. Although the paper reports per-round outcomes, an ablation without self-refinement, and early stopping, it does not establish a sufficiently clear latency–accuracy advantage over simpler tool-assisted alternatives under comparable inference budgets.
5. Incorrect supplementary code archive. The supplied supplementary ZIP contains a MedDiM medical multimodal modeling codebase rather than an identifiable Kestrel implementation. Its README and training configuration refer to MIMIC-CXR and medical multimodal training, and the proposed SAM3 grounding, claim verification, and evidence-gated refinement code could not be located in the archive. This conflicts with the paper's reproducibility statement (p. 10), which says that the full code is included in the supplementary material. Consequently, the provided archive does not allow the reported Kestrel implementation to be reproduced or checked.
Questions for authors
- What is the main methodological contribution beyond earlier multi-stage visual verification and correction methods such as Woodpecker? Which empirical results most directly isolate that contribution?
- Given the small effects in Table 3, can the authors provide uncertainty estimates or additional evidence for the benefits of evidence-gated updating and history-aware refinement?
- How are the claim-specific confidence thresholds chosen, and how does the method behave when SAM3 detections or LVLM-generated textual evidence are incorrect or incomplete?
- Can the authors provide a latency–accuracy comparison against simpler tool-assisted correction approaches under comparable inference budgets?
- The submitted supplementary ZIP appears to contain the MedDiM project rather than Kestrel. Could the authors confirm whether the wrong archive was submitted and provide the corresponding implementation?
- Figure 5's MS-COCO transition percentages imply approximately 84.3% initial accuracy and 88.1% final accuracy, whereas the corresponding Table 1 averages are approximately 87.37% and 89.45%. Were different subsets or aggregation protocols used?