- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of hallucinations in multimodal large language models (MLLMs), where correct visual information may exist in intermediate layers but hallucinations still arise in the final layer. To address this issue, the authors propose AEDG, a training-free decoding method that leverages visual attention entropy. At each decoding step, AEDG selects the layer with the lowest visual attention entropy as the positive anchor and the layer with the highest entropy as the negative anchor to calibrate the final prediction. It also employs a confidence gate and candidate token filtering to control the strength and scope of the correction. The authors report overall reductions in hallucinations while maintaining or improving multimodal performance compared with existing methods.
Critical strengths
-
AEDG selects intermediate layers based on the concentration of the visual attention distribution. Table 4 shows that this approach achieves lower hallucination scores than alternative layer selection criteria, including attention magnitude, logit entropy, and Laplacian energy, providing empirical support for entropy-based layer selection.
-
AEDG can be applied to existing MLLMs without additional training, parameter updates, or an extra forward pass. Table 5 shows that it reduces hallucinations with only a small increase in inference latency and memory usage compared with standard decoding, demonstrating its efficiency under the evaluated setting.
Critical weaknesses
-
AEDG applies corrections only to candidate tokens selected from the final layer. Therefore, if the correct token is excluded from the candidate set, it cannot be selected at that decoding step, even if relevant information remains in intermediate layers. In addition, the method uses the maximum prediction probability of the positive anchor to control its contribution, although a high prediction probability does not necessarily indicate a correct prediction. The authors acknowledge these limitations but do not analyze how frequently correct tokens are excluded from the candidate set. They also do not provide an ablation study removing the confidence gate or an empirical sensitivity analysis of the candidate filtering hyperparameters and . Consequently, it is difficult to determine how much these components individually contribute to hallucination reduction.
-
The experiments are conducted only on three models: LLaVA-1.5-7B, Qwen-VL-Chat, and GLM-4V-9B. All three belong to the 2023–2024 generation of MLLMs, and no evaluations on more recent models are provided. Therefore, it remains unclear whether entropy-based layer selection and prediction calibration can consistently improve performance on newer MLLMs, limiting the evidence for the generalizability of the proposed method.
Questions for authors
The authors state that they selected and based on the CHAIR experiments in Figure 5. However, it is unclear whether the images used in these experiments are independent of the 500 images used for the final CHAIR evaluation. Could the authors clarify whether there is any overlap between the images used for hyperparameter selection and those used for the final evaluation, and provide details about the data composition and hyperparameter selection procedure? This information is important for assessing whether the reported performance improvements generalize to data not used during hyperparameter tuning.
5 comments
Response to Reviewer V7Jt(2/2)
W2: Generalization to newer models
A: We extend CHAIR evaluation to newer-generation multimodal LLMs, InternVL3.5-8B [1] and Qwen3.5-9B [2], comparing AEDG with Vanilla, VCD, DeCo, and MemVR:
| Model | Method | CHAIR_S ↓ | CHAIR_I ↓ | Recall ↑ |
|---|---|---|---|---|
| InternVL3.5-8B | Vanilla | 36.40 | 9.56 | 71.89 |
| InternVL3.5-8B | VCD | 40.80 | 10.96 | 70.19 |
| InternVL3.5-8B | DeCo | 39.40 | 9.50 | 71.82 |
| InternVL3.5-8B | MemVR | 37.00 | 9.54 | 70.97 |
| InternVL3.5-8B | AEDG | 33.60 | 8.79 | 72.04 |
| Qwen3.5-9B | Vanilla | 62.40 | 11.74 | 79.52 |
| Qwen3.5-9B | VCD | 63.20 | 12.14 | 80.95 |
| Qwen3.5-9B | DeCo | 61.60 | 11.45 | 78.86 |
| Qwen3.5-9B | MemVR | 60.40 | 11.76 | 79.26 |
| Qwen3.5-9B | AEDG | 58.40 | 11.18 | 81.08 |
The experimental results demonstrate that AEDG consistently achieves the best CHAIR_S, CHAIR_I, and Recall metrics on these two newer-generation multimodal LLMs, strongly validating its outstanding cross-generational generalization performance and high practical value.
[1] Wang, Weiyun, et al. "InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency." arXiv preprint arXiv:2508.18265 (2025).
[2] Qwen Team. "Qwen3.5: Towards Native Multimodal Agents." Qwen Blog, 2026.
Q1: Data split and hyperparameter selection procedure
A: Hyperparameter and candidate window selection are conducted on 500 independent images sampled from MSCOCO Val2014, with no overlap between the two sets used for tuning and final CHAIR evaluation. Specifically, we conduct a grid search over the range of [0.0, 1.0] in increments of 0.1 to identify the optimal hyperparameters of and fix candidate filtering at . Our candidate layer windows are mapped via the common normalized depth interval [0.55, 0.85) to [18, 28) for LLaVA-1.5-7B/Qwen-VL-Chat and [22, 34) for GLM-4V-9B. We will revise the experimental setup to explicitly document the data composition, the non-overlapping tuning and evaluation subsets, and the hyperparameter selection procedure.
Thank you for the detailed response and additional experiments. I have reviewed the component ablations, hyperparameter sensitivity analysis, and evaluations on newer models. I also appreciate the explanation that the tuning and evaluation images do not overlap. These additions address several of my concerns regarding the experimental validation, and I have raised my rating to 2
Response to Reviewer V7Jt(1/2)
Dear Reviewer V7Jt,
Thank you for your positive assessment of entropy-guided selection and inference efficiency, and for your specific suggestions regarding candidate coverage, confidence gating, and generalization. We provide additional experiments and clarifications below.
W1: Candidate coverage, confidence gating, and filtering parameters
A: We provide candidate retention statistics, cross-model component ablations, and a filtering sensitivity analysis below.
(1) Correct-answer retention and confidently wrong predictions
The table below groups errors by baseline confidence; both retention and correction rates use all baseline errors in the corresponding group as the denominator:
The correct answer remains successfully retained for 92.51% of baseline errors, and AEDG corrects 281 high-confidence errors (achieving a 61.89% correction rate), demonstrating that our calibration algorithm possesses highly practical value in correcting confidently wrong hallucination errors.
(2) Ablations of confidence gating and candidate filtering
The confidence gate dynamically scales the calibration contributions of the anchors based on their own predicted confidence, while candidate filtering limits the correction to the retained candidate set, controlling the strength and scope of the calibration, respectively. We compare Vanilla, full AEDG, and variants that remove confidence gating or candidate filtering:
Full AEDG consistently achieves lower CHAIR_S and CHAIR_I than Vanilla across all three backbones and outperforms both ablated variants across the listed three metrics, supporting the overall calibration method and the contributions of components within the framework.
(3) Sensitivity to candidate filtering hyperparameters k,p
We vary k and p separately on LLaVA-1.5-7B to assess sensitivity to the candidate filtering settings:
Across the tested settings, CHAIR_S, CHAIR_I, and Recall span extremely narrow ranges (maximum variation of only 0.4%, 0.2%, and 0.1%), demonstrating the exceptional robustness and parameter stability of our filtering mechanism.