- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ViDRA, a visual dependency-aware attention rebalancing framework for mitigating hallucinations in Large Vision-Language Models (LVLMs). The method estimates visual token importance based on low-frequency representation stability and instruction relevance, and derives layer-wise intervention strengths from visual activity during prefill. During decoding, a lightweight Visual Dependency Gate predicts the visual dependency of upcoming tokens to dynamically adjust visual attention enhancement. Experiments across three LVLM backbones on hallucination and general vision-language benchmarks demonstrate improved hallucination mitigation while largely preserving general task performance and inference efficiency.
Critical strengths
-
Well-motivated and fine-grained attention intervention. The paper identifies the limitations of static visual attention enhancement and proposes a framework that jointly considers visual token importance, layer-wise activity, and step-wise visual dependency. The analysis in Fig. 3 provides useful evidence that visual dependency varies across generation steps and is not reliably reflected by native attention weights.
-
Strong empirical performance across multiple LVLMs. The method achieves promising results on three LVLM backbones across hallucination and general vision-language benchmarks. The component ablations and efficiency analysis further demonstrate the practical benefits of the proposed framework, particularly its ability to mitigate hallucinations without substantially degrading general task performance.
Critical weaknesses
-
Insufficient validation of the learned Visual Dependency Gate. Although the ablation study demonstrates that the Gate contributes substantially to performance improvements, it remains unclear whether these gains arise from accurate token-wise dependency prediction or simply from reducing excessive visual attention enhancement. Comparisons with simpler gating strategies under matched intervention strengths are needed to establish the specific contribution of the learned Gate.
-
Lack of direct evaluation of language generation quality. A central motivation of ViDRA is to mitigate hallucinations while preserving the model's intrinsic language generation capability. However, the general benchmarks in Table 2 primarily evaluate visual question-answering accuracy rather than language fluency, coherence, or repetition. Although qualitative examples are provided, quantitative evaluations of generation quality are needed to substantiate this central claim.
Questions for authors
-
Regarding the contribution of the Visual Dependency Gate. Could the authors compare the learned Gate with simpler alternatives, such as constant gating or token-category-based gating, under matched visual attention enhancement strengths? This would help clarify whether the improvements primarily arise from accurate visual dependency prediction or from reducing unnecessary attention intervention.
-
Regarding the preservation of language generation quality. A central motivation of ViDRA is to mitigate hallucinations while preserving language generation quality. However, the general benchmarks in Table 2 mainly evaluate visual question-answering accuracy rather than language fluency or coherence. Could the authors provide direct quantitative evaluations of language generation quality, such as fluency, repetition, and informativeness, on open-ended generation tasks? This would help substantiate the claimed trade-off between hallucination mitigation and language generation quality.
-
Regarding visual dependency supervision. The dependency labels are constructed from the log-probability differences between full-vision and vision-drop conditions, with negative values truncated to zero. How sensitive is the method to these design choices? Could the authors evaluate alternative label construction strategies to further validate the reliability of the proposed supervision signal?
-
Regarding cross-backbone generalization. Are separate Visual Dependency Gates trained for LLaVA-1.5-7B, Qwen3-VL-8B, and InternVL3.5-8B? If so, could the authors clarify the corresponding training configurations and costs? How are differences in hidden-state dimensions and feature distributions handled across these models?
-
Regarding baseline comparison fairness. In Tables 1 and 2, several attention intervention baselines, including VisFlow and PAI, underperform Greedy decoding on certain benchmarks. Could the authors clarify whether these methods were appropriately adapted to different LVLM architectures? Would tuning their intervention strengths under the same validation protocol affect the reported performance gaps?