- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a training-free framework named EViCA for adjudicating visual claims and selectively correcting LVLM outputs. It converts answers or object mentions into claims, gathers SAM 3 observations with provenance and instance associations, and judges support and refutation separately. A claim may therefore be supported, refuted, unresolved, or conflicting, and only eligible decisions are allowed to change a binary answer or remove a caption span. The paper motivates this design through three concrete failure modes, namely target mismatch, treating an empty detection as proof of absence, and deleting valid content while fixing a false mention. Evaluation is conducted on POPE, MME-Hallucination, and COCO captioning with two LVLM backbones.
Critical strengths
- The paper identifies specific decision errors rather than treating hallucination correction as a single black-box problem. In Table 1, target correspondence reduces harmful changes from 20 to 4, and combining it with preservation under empty evidence raises net correction from 14 to 31 while eliminating harmful changes in that diagnostic. This is direct evidence for the central claim that a missing detection should not automatically be interpreted as absence.
- The ablations separate the contribution of acquisition from that of adjudication. On the same acquired observations, Full avoids 204 harmful changes relative to Score only and improves accuracy by about 0.50 points; relative to Initial only, it gains about 0.69 points. The image-clustered bootstrap intervals make these comparisons more informative than a bare benchmark score.
Critical weaknesses
- The reported MME result is affected by test-set model selection. Section 5.1 and Appendix C.2 state that all 240 MME questions are used both to select the count qualification threshold and to report final performance. Table 9 shows that this choice moves the total from 730 to 740 for Qwen and from 678.33 to 695 for LLaVA. Consequently, the headline MME scores in Tables 2 and 3 are not results on an untouched test set. This matters here because thresholded, programmatic decisions are a central part of the method rather than a peripheral implementation detail.
- The main comparison does not control for method class, compute, or retained content. EViCA uses SAM 3 and several rounds of post-processing, but is compared mainly with generation-time methods such as VCD, M3ID, SID, and DiVE. There is no controlled comparison with Woodpecker, Pelican, or a budgeted evidence-acquisition method using the same backbone and observation budget. EViCA also costs 21,806 MB and 77.7 ms/token versus 17,568 MB and 33.4 ms/token for the base model (Table 21). On captioning, lower CHAIR comes with recall drops from 71.70 to 65.87 for Qwen and from 80.50 to 75.82 for LLaVA. Without a broader quality assessment, some of the gain may reflect more aggressive deletion rather than a uniformly better caption. Finally, the reproducibility statement says that the core EViCA implementation will remain closed, even though route checks, priorities, query normalization, and deletion eligibility are crucial to the result.
Questions for authors
- Can the authors select every POPE and MME threshold on a separate validation set and report MME results, with confidence intervals, on data that played no role in model selection?
- How does EViCA compare with Woodpecker, Pelican, and at least one budgeted acquisition method when the backbone, SAM 3 observations, number of tool calls, and latency budget are matched?
- Will the authors release the core implementation? If not, can they release all prompts, route conditions, action priorities, calibration splits, random seeds, and per-example adjudication records needed to reproduce the system?
- Does the captioning advantage remain under a paired human evaluation that jointly considers factuality, completeness, and fluency rather than CHAIR alone?
- On p. 9, the Figure 3 caption says that “stars mark the selected thresholds,” but no star markers are visible in panels (b) or (c). Please add the markers or remove this sentence.
- In the reference title on p. 11, “adaptive” in “Global context or local detail? adaptive visual grounding...” should be capitalized.
- The reference title on p. 13 ends with “vision-language model”; the published title uses the plural “vision-language models.”
- Figures 1 and 5 use “SAM3,” whereas the main text and Figure 2 use “SAM 3.” Please use one form consistently.