- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes CIVER, a training-free method for mitigating hallucinations in large vision-language models. CIVER integrates a sample-specific visual prior, candidate-conditioned supporting and competing evidence, and cross-step evidence memory to selectively adjust visual attention during decoding. Experiments across four LVLMs on POPE, CHAIR, MME, and AMBER demonstrate improvements in hallucination mitigation over existing baselines.
Critical strengths
The paper presents a valuable observation that hallucinations are more likely to occur when the visual evidence distributions associated with competing token candidates are difficult to distinguish. This finding offers a new perspective on understanding and detecting hallucinations in LVLMs.
Critical weaknesses
- The connection between the research motivation and the core intervention mechanism is insufficiently justified. The paper argues that hallucinations are more likely to occur when the visual evidence distributions of competing candidates exhibit substantial overlap, making them difficult to distinguish. Following this motivation, the intervention should help the model more reliably distinguish competing candidates based on visual evidence. However, CIVER enhances the visual evidence associated with the top-ranked candidate while suppressing evidence that uniquely favors the second-ranked candidate. This design appears to implicitly assume that the leading candidate is more reliable, which may not hold in hallucination-prone scenarios, where the leading candidate itself could be incorrect.
- Insufficient justification for the candidate-conditioned visual evidence estimation. The paper estimates candidate-specific visual evidence distributions by computing similarities between candidate-conditioned queries and visual token representations. However, this design relies heavily on heuristic choices and lacks sufficient justification. For example, the candidate-conditioned query is constructed by combining the anchor-layer hidden state with the corresponding candidate's LM Head output weight vector, but it remains unclear whether the dot product between the resulting query and visual token representations can reliably capture the visual evidence supporting that candidate. Moreover, since both and share the same contextual hidden state , the overlap between their estimated visual evidence distributions may partly arise from the query construction itself, rather than genuinely reflecting the model's ability to discriminate between candidates based on visual information.
- The evaluated models are relatively outdated, leaving the applicability to recent LVLMs unclear.
Questions for authors
no other questions