- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AMEND, a training-free decoding strategy that mitigates hallucinations in LVLMs by dynamically recalibrating visual attention. It consists of two components. Uncertainty-Aware Adaptive Scaling adjusts the visual attention enhancement coefficient based on the entropy of the output logits. Expert-Layer Attention-Driven Token Extraction selects question-relevant visual tokens from vision-sensitive expert layers. Attention to the selected tokens is amplified, while attention to the other visual tokens and the system prompt is suppressed. Experiments on POPE and MME-Hallucination with LLaVA-1.5 and Qwen2.5-VL show that AMEND outperforms existing decoding-based methods in most settings.
Critical strengths
- Modulating visual attention according to model uncertainty is a reasonable idea. AMEND decides both when to intervene (via entropy) and where to intervene (via critical tokens), which avoids the over- or under-enhancement of fixed-coefficient methods.
- The method is training-free and applicable to different LVLMs (LLaVA-1.5 and Qwen2.5-VL). It is relatively efficient compared with contrastive decoding baselines and achieves strong results across multiple benchmarks.
Critical weaknesses
- The novelty is limited. Uncertainty-triggered visual intervention and selective manipulation of visual attention have already been explored, e.g., in MemVR (ICML 2025), AdaptVis (ICML 2025), VAR (ICLR 2025), and Self-Aug (ICLR 2026). The differences from these works should be clarified.
- The method has many hyperparameters (entropy center and scale, coefficient bounds, β, γ, λ, and the layer range), and the paper states that layer selection and the token retention ratio are tuned per task type. This raises concerns about generalization to new models and tasks.
- The assumption that high entropy indicates hallucination is supported only by an empirical analysis of a single model on a single benchmark (Fig. 2), without theoretical justification. High entropy can also stem from linguistic ambiguity or inherently uncertain predictions, and hallucinations can occur with high confidence.
- It is unclear whether high-attention tokens in the expert layers actually correspond to question-relevant objects. Prior work shows that LVLMs often assign high attention to irrelevant background tokens (e.g., Visual Attention Sink, ICLR 2025; DAMRO, EMNLP 2024), so amplifying such tokens may reinforce hallucinations rather than mitigate them.
Questions for authors
- Could you provide a quantitative analysis of whether the selected tokens focus on meaningful regions, for example their overlap with ground-truth object regions, or a comparison with randomly selected tokens?
- Standard attention weights sum to one after softmax, but multiplying them by coefficients as in Eq. (11) may break this property. Is the modified attention renormalized? If not, how does this affect the model?
- Could you evaluate AMEND on open-ended generation benchmarks such as CHAIR, where token-level adaptivity should matter more?