- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates visual hallucination in multimodal large language models (MLLMs) from the perspective of layer-wise evidence competition. The authors identify a failure mode termed strict silencing, where intermediate layers favor the factual answer but the final prediction favors an incorrect alternative. To address this issue, the paper proposes Adaptive Counter-context Evidence (ACE), a training-free decoding framework that constructs controlled counter-context probes, estimates visual contribution reliability, locates an evidence anchor, and selectively amplifies visual contributions throughout the decoder suffix. Experiments across four MLLM backbones show substantial improvements on POPE, CHAIR, and FREAK while largely preserving MM-Vet performance. Additional functional diagnostics and ablation studies support the effectiveness of reliability-based evidence selection and sustained preservation.
Critical strengths
-
Interesting motivation and evidence analysis. The paper provides a useful perspective on visual hallucination by examining how intermediate factual dominance can be lost during decoding. The strict-silencing formulation and layer-wise diagnostics help characterize this failure mode more precisely.
-
Strong empirical results with extensive validation. ACE consistently improves hallucination-related benchmarks across four MLLM backbones. The functional evidence diagnostics, preservation ablations, and amplitude-matched comparisons provide meaningful support for the proposed reliability-based intervention.
Critical weaknesses
-
The connection between strict silencing and the overall performance gains remains incomplete. Although strict silencing motivates the proposed framework, a substantial portion of ACE's corrections occur outside the strict-silencing cohort. For example, Table 14 shows that 120 of the 191 corrected errors on Qwen3-VL are outside this cohort. This raises questions about whether the observed improvements are primarily attributable to recovering silenced evidence.
-
The relationship between reliability estimation and factual improvement remains insufficiently established. ACE identifies reliable visual contributions based on counter-context stability and visual specificity, but these properties do not necessarily imply factual correctness. Although the AFE diagnostics support the effectiveness of the proposed ranking, single-unit deletion effects may not directly reflect the outcomes of sustained amplification across multiple decoder layers. Therefore, it remains unclear how reliably the estimated scores distinguish beneficial visual contributions from stable but potentially misleading ones.
Questions for authors
-
Regarding the relationship between strict silencing and overall improvements. Table 14 shows that a substantial fraction of corrected errors fall outside the strict-silencing cohort. Could the authors further characterize these cases and clarify whether ACE primarily addresses strict silencing or a broader visual-evidence underutilization problem?
-
Regarding reliability estimation and functional evidence. The proposed reliability scores are validated using single-unit ablation effects, whereas ACE applies sustained amplification to multiple visual contributions. How well do these scores predict the actual effects of amplification? In particular, what prevents ACE from amplifying stable but factually incorrect visual contributions?
-
Regarding prediction stability and generalization. Table 14 reports zero correct-to-incorrect transitions across all four POPE-Adversarial cohorts. Could the authors provide further analysis of this result and investigate whether similar prediction stability holds across other benchmarks, challenging visual conditions, and different preservation strengths?
-
Regarding counter-context robustness. How sensitive is ACE to the size and semantic coverage of the counter-context bank and to the placement of contextual edits? In particular, how does the method behave when the edited region contains relevant visual information or when no explicit target concept is available?
-
Regarding manuscript and figure quality. The manuscript would benefit from more careful editing and formatting. Several figures, particularly Figures 1 and 2, contain overly compressed elements, crowded annotations, and unclear visual connections. In addition, inconsistent spacing and excessive whitespace affect the overall presentation. Although the authors disclose the use of AI tools for manuscript and figure preparation, the current presentation still requires substantial manual refinement. Could the authors improve the figure layouts, arrow alignment, text readability, and document formatting?