acceptodds
Under review as a conference paper at ICLR 2027

Attention-Entropy Decoding with Visual Grounding for Hallucination Mitigation in Multimodal Large Language Models

Abstract

Multimodal large language models (MLLMs) have made significant progress in vision–language reasoning, yet they remain prone to hallucination even when relevant visual information is already present in intermediate layers. Existing mitigation methods either require additional training or address the problem during decoding through visual reinforcement, contrastive prediction, or layer-wise correction. However, accurately identifying which intermediate layer contains useful visual information remains challenging. We hypothesize that the concentration of visual attention provides an effective signal: low-entropy attention indicates concentrated and reliable visual information, whereas high-entropy attention suggests dispersed and unreliable information. Based on this insight, we propose AEDG, a training-free attention-entropy decoding framework that uses visual information to guide the decoding process. At each decoding step, AEDG selects a low-entropy layer as a positive anchor and a high-entropy layer as a negative anchor, and uses them to adjust the final-layer prediction. AEDG requires neither parameter updates nor additional forward computation. Extensive experiments on representative MLLMs show that AEDG effectively reduces hallucination while maintaining strong multimodal capability with only a small inference overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Visual Hallucination, Visual Grounding, Attention Entropy, Contrastive Decoding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_V7JtOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of hallucinations in multimodal large language models (MLLMs), where correct visual information may exist in intermediate layers but hallucinations still arise in the final layer. To address this issue, the authors propose AEDG, a training-free decoding method that leverages visual attention entropy. At each decoding step, AEDG selects the layer with the lowest visual attention entropy as the positive anchor and the layer with the highest entropy as the negative anchor to calibrate the final prediction. It also employs a confidence gate and candidate token filtering to control the strength and scope of the correction. The authors report overall reductions in hallucinations while maintaining or improving multimodal performance compared with existing methods.

Critical strengths
  1. AEDG selects intermediate layers based on the concentration of the visual attention distribution. Table 4 shows that this approach achieves lower hallucination scores than alternative layer selection criteria, including attention magnitude, logit entropy, and Laplacian energy, providing empirical support for entropy-based layer selection.

  2. AEDG can be applied to existing MLLMs without additional training, parameter updates, or an extra forward pass. Table 5 shows that it reduces hallucinations with only a small increase in inference latency and memory usage compared with standard decoding, demonstrating its efficiency under the evaluated setting.

Critical weaknesses
  1. AEDG applies corrections only to candidate tokens selected from the final layer. Therefore, if the correct token is excluded from the candidate set, it cannot be selected at that decoding step, even if relevant information remains in intermediate layers. In addition, the method uses the maximum prediction probability of the positive anchor to control its contribution, although a high prediction probability does not necessarily indicate a correct prediction. The authors acknowledge these limitations but do not analyze how frequently correct tokens are excluded from the candidate set. They also do not provide an ablation study removing the confidence gate or an empirical sensitivity analysis of the candidate filtering hyperparameters pp and kk. Consequently, it is difficult to determine how much these components individually contribute to hallucination reduction.

  2. The experiments are conducted only on three models: LLaVA-1.5-7B, Qwen-VL-Chat, and GLM-4V-9B. All three belong to the 2023–2024 generation of MLLMs, and no evaluations on more recent models are provided. Therefore, it remains unclear whether entropy-based layer selection and prediction calibration can consistently improve performance on newer MLLMs, limiting the evidence for the generalizability of the proposed method.

Questions for authors

The authors state that they selected α=0.1\alpha=0.1 and β=0.6\beta=0.6 based on the CHAIR experiments in Figure 5. However, it is unclear whether the images used in these experiments are independent of the 500 images used for the final CHAIR evaluation. Could the authors clarify whether there is any overlap between the images used for hyperparameter selection and those used for the final evaluation, and provide details about the data composition and hyperparameter selection procedure? This information is important for assessing whether the reported performance improvements generalize to data not used during hyperparameter tuning.

5 comments
AuthorsOpenReview ↗

Response to Reviewer V7Jt(1/2)

Dear Reviewer V7Jt,

Thank you for your positive assessment of entropy-guided selection and inference efficiency, and for your specific suggestions regarding candidate coverage, confidence gating, and generalization. We provide additional experiments and clarifications below.

W1: Candidate coverage, confidence gating, and filtering parameters

A: We provide candidate retention statistics, cross-model component ablations, and a filtering sensitivity analysis below.

(1) Correct-answer retention and confidently wrong predictions

The table below groups errors by baseline confidence; both retention and correction rates use all baseline errors in the corresponding group as the denominator:

Baseline error groupNumber of errorsErrors retaining the correct answerRetention rate (%)Errors corrected by AEDGCorrection rate (%)
Low confidence: c<0.8c<0.81,3611,361100.0097971.93
High confidence: c≥0.8c\geq0.845431870.0428161.89
All baseline errors1,8151,67992.511,26069.42

The correct answer remains successfully retained for 92.51% of baseline errors, and AEDG corrects 281 high-confidence errors (achieving a 61.89% correction rate), demonstrating that our calibration algorithm possesses highly practical value in correcting confidently wrong hallucination errors.

(2) Ablations of confidence gating and candidate filtering

The confidence gate dynamically scales the calibration contributions of the anchors based on their own predicted confidence, while candidate filtering limits the correction to the retained candidate set, controlling the strength and scope of the calibration, respectively. We compare Vanilla, full AEDG, and variants that remove confidence gating or candidate filtering:

ModelVariantCHAIR_S ↓CHAIR_I ↓Recall ↑
LLaVA-1.5-7BVanilla50.015.477.1
LLaVA-1.5-7BWithout confidence gating43.211.776.4
LLaVA-1.5-7BWithout candidate filtering43.612.176.9
LLaVA-1.5-7BFull AEDG39.810.677.2
Qwen-VL-ChatVanilla6.85.353.4
Qwen-VL-ChatWithout confidence gating5.43.651.9
Qwen-VL-ChatWithout candidate filtering5.23.152.4
Qwen-VL-ChatFull AEDG5.03.052.6
GLM-4V-9BVanilla41.88.773.5
GLM-4V-9BWithout confidence gating40.28.772.4
GLM-4V-9BWithout candidate filtering40.49.172.6
GLM-4V-9BFull AEDG39.28.573.7

Full AEDG consistently achieves lower CHAIR_S and CHAIR_I than Vanilla across all three backbones and outperforms both ablated variants across the listed three metrics, supporting the overall calibration method and the contributions of components within the framework.

(3) Sensitivity to candidate filtering hyperparameters k,pk,p

We vary kk and pp separately on LLaVA-1.5-7B to assess sensitivity to the candidate filtering settings:

ModelkpCHAIR_S ↓CHAIR_I ↓Recall ↑
LLaVA-1.5-7B50.9040.210.877.3
LLaVA-1.5-7B100.9039.810.677.2
LLaVA-1.5-7B200.9040.010.777.2
LLaVA-1.5-7B100.8540.210.877.3
LLaVA-1.5-7B100.9539.810.877.2

Across the tested settings, CHAIR_S, CHAIR_I, and Recall span extremely narrow ranges (maximum variation of only 0.4%, 0.2%, and 0.1%), demonstrating the exceptional robustness and parameter stability of our filtering mechanism.

AuthorsOpenReview ↗

Response to Reviewer V7Jt(2/2)

W2: Generalization to newer models

A: We extend CHAIR evaluation to newer-generation multimodal LLMs, InternVL3.5-8B [1] and Qwen3.5-9B [2], comparing AEDG with Vanilla, VCD, DeCo, and MemVR:

ModelMethodCHAIR_S ↓CHAIR_I ↓Recall ↑
InternVL3.5-8BVanilla36.409.5671.89
InternVL3.5-8BVCD40.8010.9670.19
InternVL3.5-8BDeCo39.409.5071.82
InternVL3.5-8BMemVR37.009.5470.97
InternVL3.5-8BAEDG33.608.7972.04
Qwen3.5-9BVanilla62.4011.7479.52
Qwen3.5-9BVCD63.2012.1480.95
Qwen3.5-9BDeCo61.6011.4578.86
Qwen3.5-9BMemVR60.4011.7679.26
Qwen3.5-9BAEDG58.4011.1881.08

The experimental results demonstrate that AEDG consistently achieves the best CHAIR_S, CHAIR_I, and Recall metrics on these two newer-generation multimodal LLMs, strongly validating its outstanding cross-generational generalization performance and high practical value.

[1] Wang, Weiyun, et al. "InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency." arXiv preprint arXiv:2508.18265 (2025).

[2] Qwen Team. "Qwen3.5: Towards Native Multimodal Agents." Qwen Blog, 2026.

Q1: Data split and hyperparameter selection procedure

A: Hyperparameter and candidate window selection are conducted on 500 independent images sampled from MSCOCO Val2014, with no overlap between the two sets used for tuning and final CHAIR evaluation. Specifically, we conduct a grid search over the range of [0.0, 1.0] in increments of 0.1 to identify the optimal hyperparameters of α=0.1,β=0.6\alpha=0.1, \beta=0.6 and fix candidate filtering at k=10,p=0.9k=10, p=0.9. Our candidate layer windows are mapped via the common normalized depth interval [0.55, 0.85) to [18, 28) for LLaVA-1.5-7B/Qwen-VL-Chat and [22, 34) for GLM-4V-9B. We will revise the experimental setup to explicitly document the data composition, the non-overlapping tuning and evaluation subsets, and the hyperparameter selection procedure.

Thank you for the detailed response and additional experiments. I have reviewed the component ablations, hyperparameter sensitivity analysis, and evaluations on newer models. I also appreciate the explanation that the tuning and evaluation images do not overlap. These additions address several of my concerns regarding the experimental validation, and I have raised my rating to 2

Reviewer_gudUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes AEDG, a training-free decoding method for reducing hallucinations in multimodal large language models. At each decoding step, AEDG uses visual-attention entropy to select two intermediate layers. The lowest-entropy layer serves as a positive anchor, while the highest-entropy layer serves as a negative anchor. Their logits are combined with the final-layer prediction through confidence-aware calibration and candidate filtering.This paper also reports improvements on CHAIR, POPE, HallusionBench, and MME using LLaVA-1.5-7B, Qwen-VL-Chat, and GLM-4V-9B.

Critical strengths
  1. AEDG is computationally efficient. AEDG uses attention structure to select intermediate layers. The dual-anchor design requires no training or additional forward pass. On LLaVA-1.5-7B, latency increases only from 31.0 to 31.8 ms per token.
  2. The object-hallucination results are strong. AEDG achieves the best CHAIR results across all three tested models and performs well across all POPE splits. The paper also compares several layer-selection signals and includes fixed, random, and reversed-anchor controls. These ablations provide useful evidence that both layer selection and anchor direction matter.
Critical weaknesses
  1. Some important implementation details are missing. The authors state that the candidate layer window is adjusted for each backbone, but they do not report the window used for each model or explain how it was selected. The paper also does not say whether the hyperparameters in Figure 5 were chosen on a separate validation set. In addition, the confidence gate and candidate filtering are not ablated.
  2. The ablation study is limited to LLaVA-1.5-7B on CHAIR. It is therefore unclear whether the same design choices work equally well for Qwen-VL-Chat and GLM-4V-9B. This matters because some improvements on these models are small, and AEDG is slightly worse than the base model on a few HallusionBench metrics. Confidence intervals or paired tests would help determine whether these differences are meaningful.
Questions for authors
  1. What candidate layer window is used for each backbone? Were the layer windows and other hyperparameters selected on a separate validation set?
  2. Can the authors report ablations for the confidence gate and candidate filtering?
4 comments
AuthorsOpenReview ↗

Response to Reviewer gudU

Dear Reviewer gudU,

Thank you for your positive assessment of AEDG's efficiency and hallucination mitigation results, and for your suggestions on implementation details, cross-model ablations, and statistical analysis. We address each concern below.

W1: Implementation details and hyperparameter selection

A: All candidate layer windows and hyperparameters are determined based on CHAIR metrics evaluated on 500 independent MSCOCO Val2014 validation images, with no overlap between the two sets used for tuning and final evaluation. Specifically, we conduct a grid search for α\alpha and β\beta over the range of [0.0, 1.0] with a step size of 0.1, identifying the optimal settings of α=0.1,β=0.6\alpha=0.1, \beta=0.6 and candidate filtering fixed at k=10,p=0.9k=10, p=0.9. Based on these tuning metrics, the final candidate windows (all left-closed and right-open) are set to [18, 28) for LLaVA-1.5-7B and Qwen-VL-Chat, and [22, 34) for GLM-4V-9B. Ablations of gating and filtering are provided in Q2.

W2: Cross-model ablations and HallusionBench statistical analysis

A: To fully validate our design choices across backbones, we extend our gating and filtering ablations to Qwen-VL-Chat and GLM-4V-9B (see Q2). Additionally, to demonstrate the statistical significance of our gains, we report differences between AEDG and Vanilla on HallusionBench along with paired 95% confidence intervals (in percentage points):

ModelComparisonΔfACC [95% CI]ΔqACC [95% CI]ΔeasyA [95% CI]ΔhardA [95% CI]ΔaACC [95% CI]
LLaVA-1.5-7BAEDG − Vanilla-1.1 [-1.7, -0.4]+2.7 [+2.1, +3.4]+4.4 [+3.3, +5.5]+2.8 [+2.0, +3.6]+2.0 [+1.2, +2.8]
Qwen-VL-ChatAEDG − Vanilla+2.9 [+2.2, +3.6]+2.4 [+1.6, +3.2]+2.7 [+2.0, +3.1]+1.5 [+0.9, +2.1]+1.9 [+1.0, +2.8]
GLM-4V-9BAEDG − Vanilla+1.5 [+0.8, +2.2]+0.9 [+0.3, +1.5]-0.4 [-0.9, +0.4]+2.4 [+1.7, +3.1]+0.5 [+0.1, +0.9]

The confidence intervals of our primary metric aACC are strictly above zero for all three backbones, proving the statistical significance of AEDG's improvements in factual judgment. The overall improvements demonstrate robust and statistically significant factual gains.

Q1: Independence of tuning and final evaluation data

A: Yes. The 500 images used for hyperparameter and window selection are strictly non-overlapping with the separate 500 images used for the final CHAIR evaluation, ensuring the reliability of our generalization results without data leakage.

Q2: Cross-model ablations of gating and candidate filtering

A: We compare Vanilla, full AEDG, and ablated variants that remove confidence gating or candidate filtering across three backbones:

ModelVariantCHAIR_S ↓CHAIR_I ↓Recall ↑
LLaVA-1.5-7BVanilla50.015.477.1
LLaVA-1.5-7BWithout confidence gating43.211.776.4
LLaVA-1.5-7BWithout candidate filtering43.612.176.9
LLaVA-1.5-7BFull AEDG39.810.677.2
Qwen-VL-ChatVanilla6.85.353.4
Qwen-VL-ChatWithout confidence gating5.43.651.9
Qwen-VL-ChatWithout candidate filtering5.23.152.4
Qwen-VL-ChatFull AEDG5.03.052.6
GLM-4V-9BVanilla41.88.773.5
GLM-4V-9BWithout confidence gating40.28.772.4
GLM-4V-9BWithout candidate filtering40.49.172.6
GLM-4V-9BFull AEDG39.28.573.7

Compared with Vanilla, full AEDG consistently reduces CHAIR_S and CHAIR_I across all backbones, while outperforming both ablated variants across the listed three metrics, supporting the contributions of gating and filtering to the overall method. Relative to Vanilla, Recall increases slightly on LLaVA and GLM and decreases slightly on Qwen.

AuthorsOpenReview ↗

Kindly Request for Feedback of Reviewer gudU

Dear Reviewer gudU,

Thank you for your thoughtful feedback. As the discussion period draws to a close, we would appreciate your feedback on whether our responses and additional experiments have addressed your main concerns. If any concerns remain, we would be happy to provide further clarification. If your concerns have been resolved, we would appreciate your reconsideration of the rating in light of the additional evidence.

Best regards, The Authors

Reviewer_gudUOpenReview ↗

Official Comment by Reviewer gudU

Thank you for the additional experiments. These results have addressed some of my concerns.I'll keep my positive rating.

Dear Reviewer gudU,

Thank you for your feedback and for maintaining your positive rating. We are glad that the additional experiments have addressed your concerns.

Best regards, The Authors

Reviewer_T3ejOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes AEDG, which is a useful, free training framework to reduce hallucinations in multimodal large language models. It selects low and high entropy visual attention layers at each decoding step and uses their logits to adjust the final prediction. The author evaluates the method across multiple model families and several hallucination benchmarks, showing decent improvement with small inference overhead they report.

Critical strengths
  1. The method is simple yet effective, intuitive, and easy to follow. The idea is well motivated and tarkled an important problem in multimodal models. I think the insight about the role of the attention-entropy is very interesting and thoughtful. The theoretical analysis also helps explain the design and the conditions in a solid way (though I'm not an expert in the theory part, so I might not fully follow all the proof).

  2. The proposed results show substantial improvements over the several strong baselines, especially on selection and multimodal questions. The paper also includes useful comparisons of the condition representation and retrieval policy. The authors also provide a comprehensive comparison of the layer selection signal, anchor choice, and other design factors. The result supports their design choice well.

Critical weaknesses
  1. In many sections of the paper, the analysis show that, within a given layer, questions with lower entropy are more likely to be answered correctly. But the method's assumption is actually built on: lower-entropy layers to be more reliable for the same question. The assumption and the question mainly prove they are corresponding but might not directly imply the other. This is not to say there is no experimental support, but the key mechanism might need furthur stronger evidence and proof.

  2. The improvement on LLaVA is useful, but several gains over the strongest baseline on the other backbones are small. For example, the change from 5.2 to 5.0 CHAIR_S on Qwen means roughly just one caption in the 500 image evaluation set. Meanwhile, the layer window is adjusted empirically for each backbone, but the exact windows and the data used to select them are not clearly claim. So it might be a little difficult to assess how well the selection rule transfers across models and how stable the smaller improvements are.

Questions for authors
  1. Do the selected anchors actually provide the evidence for the correction? For the same question and decoding step, I was curious how well does visual attention entropy rank the layers by their support for the correct answer? More specific: how often does adding the positive anchor push the answer to correct side, and how often does subtracting the negative anchor improve it? I think might be helpful to report this separately for Yes / No questions

  2. AEDG selects the lowest and highest entropy layers at every step, but these are (if I understand correctly) relative choices within the candidate window. So what happens when neither layer provides useful visual evidence?

3 Because the fix lies on the final layer, the correction is restricted to the final layer candidate set. If the correct token has already been excluded, neither anchor can recover it in this case. In some case the model can be confident to the prediction and leave only wrong answers at the final layer, so what fraction of baseline errors still retain the correct/wrong answer in this case? Do the gains mainly come from uncertain predictions, or also from confidently wrong ones?

  1. How sensitive is AEDG to the candidate layer window across model sizes and families? Might be benefit to have some ablation study acorss model sizes, and also it might be helpful to report the detailed selection method and check whether normalized depth window will have the similar outcome.
5 comments
AuthorsOpenReview ↗

Response to Reviewer T3ej(1/2)

Dear Reviewer T3ej,

Thank you for your positive assessment of the AEDG design and experimental results, and for your specific suggestions regarding mechanistic evidence and result stability. We address each concern below.

W1: Layer reliability within the same question

A: We agree with your view that "layer reliability within the same question" is the most crucial evidence to validate our method's hypothesis. To directly verify this, we conduct a per-question analysis on POPE: with the prefix and decoding step fixed, we compute the Spearman correlation between visual attention entropy and the correct-answer margin across candidate layers, then average these correlations separately for Yes and No questions. The margin is the layer's score for the correct answer minus its score for the incorrect answer. We also compare predictions from the lowest-entropy, highest-entropy, and uniformly random layers:

Ground-truth answerMean Spearman ρ(H, margin) [95% CI]Lowest-entropy layer Acc (%) ↑Highest-entropy layer Acc (%) ↑Uniformly random layer Acc (%) ↑Vanilla Acc (%) ↑
Yes-0.732 [-0.749, -0.714]86.5674.9676.9676.84
No-0.714 [-0.726, -0.702]84.0478.2482.6482.82
All-0.723 [-0.739, -0.707]85.3076.6079.8079.83

The experimental results show that within the same question, there is a significant strong negative correlation between visual attention entropy and the correct-answer margin (overall ρ = -0.723), indicating that layers with more concentrated attention (lower entropy) provide higher support for the correct answer. Furthermore, directly using the lowest-entropy layer yields an accuracy of 85.30%, which substantially outperforms both the highest-entropy layer (76.60%) and the random layer (79.80%). This provides the most direct empirical evidence for our attention entropy-driven layer selection mechanism.

W2: Cross-backbone gains and evaluation on newer models

A: On Qwen-VL-Chat, Vanilla, MemVR, and AEDG achieve CHAIR_S scores of 6.8%, 5.2%, and 5.0%, respectively. AEDG reduces CHAIR_S by 1.8 percentage points relative to Vanilla, corresponding to a 26.5% relative reduction. To further assess its applicability to newer models, we conduct additional CHAIR experiments on InternVL3.5-8B [1] and Qwen3.5-9B [2]. On both new backbones, AEDG reduces CHAIR_S and CHAIR_I and improves Recall relative to Vanilla. The table below summarizes CHAIR_S changes across the five evaluated backbones:

Backbone ModelModel SizeVanilla CHAIR_SAEDG CHAIR_SAbsolute Change (pp)Relative Change (%)
LLaVA-1.5-7B7B50.0%39.8%-10.2-20.4%
Qwen-VL-Chat7B6.8%5.0%-1.8-26.5%
GLM-4V-9B9B41.8%39.2%-2.6-6.2%
InternVL3.5-8B8B36.4%33.6%-2.8-7.7%
Qwen3.5-9B9B62.4%58.4%-4.0-6.4%

AEDG reduces CHAIR_S on all five evaluated backbones, providing additional evidence of its cross-model applicability.

[1] Wang, Weiyun, et al. "InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency." arXiv preprint arXiv:2508.18265 (2025).

[2] Qwen Team. "Qwen3.5: Towards Native Multimodal Agents." Qwen Blog, 2026.

Q1: Contributions of the correction terms and Yes/No results

A: We provide ablations retaining only the high-entropy-related correction term or only the low-entropy positive term, with Yes/No results compared against Vanilla and full AEDG:

VariantYes Acc (%) ↑No Acc (%) ↑All Acc (%) ↑
Vanilla76.8482.8279.83
High-entropy-related correction term only83.6285.4784.54
Low-entropy positive term only85.2784.5184.89
Full AEDG87.7887.1687.47

Both single-term variants consistently improve Yes/No accuracy, while the full AEDG achieves the best performance across all categories and overall. This clearly validates the outstanding synergistic and complementary roles of combining the low-entropy positive and high-entropy negative anchors.

AuthorsOpenReview ↗

Response to Reviewer T3ej(2/2)

Q2: When neither anchor provides useful visual evidence

A: To address this, we adopt anchor confidence gating: correcting the final-layer logits based on predicted confidence to dynamically scale the calibration contributions of anchor, with candidate filtering limiting the calibration scope. If anchors' attention maps lack effective visual focus (exhibiting low confidence), the gating mechanism automatically decays their contributions. This allows the decoding algorithm to safely and smoothly fall back to final-layer logits predictions, ensuring the safety margin of correction by design.

Q3: Correct-answer retention and confidently wrong predictions

A: We acknowledge this limitation: if the correct token is excluded from the final-layer candidate set, neither anchor can recover it at that decoding step. To quantify the impact of this restriction and examine AEDG's ability to correct low- and high-confidence errors, we analyze baseline errors grouped by confidence, reporting correct-answer retention and correction rates.

Baseline error groupNumber of errorsErrors retaining the correct answerRetention rate (%)Errors corrected by AEDGCorrection rate (%)
Low confidence: c<0.8c<0.81,3611,361100.0097971.93
High confidence: c≥0.8c\geq0.845431870.0428161.89
All baseline errors1,8151,67992.511,26069.42

The results show that: (1) The correct answer remains in the candidate set for 92.51% of baseline errors, offering ample margin for decoding correction. (2) AEDG successfully corrects low-confidence errors (71.93% correction rate) and high-confidence errors (61.89% correction rate). This demonstrates that our method is not limited to resolving uncertain predictions, but effectively breaks through and corrects confidently wrong, stubborn hallucinations.

Q4: Window selection and cross-model sensitivity

A: The candidate windows of all three backbones are selected using CHAIR metrics on a separate set of 500 MSCOCO Val2014 images. These windows can be expressed using a highly consistent normalized depth interval of [0.55, 0.85): multiplying both boundaries by each model's total layers and rounding upward yields [18, 28) for LLaVA-1.5-7B and Qwen-VL-Chat, and [22, 34) for GLM-4V-9B. Sensitivity results for alternative windows are reported below:

ModelCandidate windowCHAIR_S (%) ↓CHAIR_I (%) ↓Recall (%) ↑
LLaVA-1.5-7B[18,28)39.810.677.2
LLaVA-1.5-7B[12,25)40.410.877.6
Qwen-VL-Chat[18,28)5.03.052.6
Qwen-VL-Chat[12,25)5.23.353.2
GLM-4V-9B[16,28)39.68.973.9
GLM-4V-9B[22,34)39.28.573.7

The small metric changes across the tested candidate windows indicate limited sensitivity to window selection on the evaluated backbones.

AuthorsOpenReview ↗

Kindly Request for Feedback of Reviewer T3ej

Dear Reviewer T3ej,

Thank you for your thoughtful feedback. As the discussion period draws to a close, we would appreciate your feedback on whether our responses and additional experiments have addressed your main concerns. If any concerns remain, we would be happy to provide further clarification. If your concerns have been resolved, we would appreciate your reconsideration of the rating in light of the additional evidence.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear AC and Reviewers,

We thank you for your careful reviews and constructive suggestions. We appreciate the positive comments, including “simple yet effective, intuitive, and easy to follow,” “very interesting and thoughtful,” and “substantial improvements” (T3ej); “computationally efficient” and “the object-hallucination results are strong” (gudU); and “empirical support for entropy-based layer selection” (V7Jt).

During the rebuttal, we carefully answered the reviewers’ questions and concerns point by point.

  • T3ej: Stated that the additional results “resolve most of my concerns” and raised the rating from 3 to 4.
  • gudU: Wrote, “These results have addressed some of my concerns. I'll keep my positive rating,” maintaining a positive rating of 3.
  • V7Jt: Acknowledged that the additions “address several of my concerns regarding the experimental validation,” raised the rating from 1 to 2, and did not raise any further concerns in the follow-up comment.

Overall, we believe that this rebuttal reflects a satisfactory response to the reviewers’ comments. We will include the additional results and experimental details in the revised manuscript. Thank you for your consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.