- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses visual hallucination mitigation in LVLMs by showing that separately estimating correction directions for object, attribute, and relation hallucinations and then fusing them is more effective than learning a single direction from mixed categories. The proposed HCA-Edit normalizes and combines category-specific directions and uses category-aware layer gating to adapt the editing strength across layers. Experiments across multiple LVLMs and benchmarks show consistent hallucination reduction while largely preserving general visual capabilities.
Critical strengths
The paper addresses the important and practically relevant problem of hallucination mitigation in LVLMs, while considering multiple hallucination categories beyond object-level hallucinations.
Critical weaknesses
-
LURE-3C is presented as a major contribution and serves as the basis of the proposed method. However, its construction largely extends the paired-data workflow used by Nullu from object hallucinations to attribute and relation hallucinations. The paper also reports no human annotation or systematic validation of the generated category labels, limiting both its novelty and reliability.
-
The effectiveness of the category-balanced direction construction is not well supported by the ablation results. In Table 4, “Category-balanced + Uniform” performs worse than “Pooled + Uniform,” leaving the independent benefit of category-specific direction estimation unclear and casting doubt on the stated motivation for estimating category-specific directions before combining them. Moreover, the main evaluations are object-centric (CHAIR and POPE), while the limited three-category evaluation reports increased attribute hallucination severity (Color sub-task in MME and Attribute dimension of LLaVA-Bench), providing insufficient evidence that the method benefits all three hallucination categories.
Questions for authors
-
What is the motivation for introducing the floor in Eq. (7) and the additional editing strength in Eq. (8)? Please clarify why both components are necessary and provide ablations removing them.
-
Although three LVLMs are considered, POPE is evaluated only on the two LLaVA models, while MME and LLaVA-Bench are evaluated only on LLaVA-1.5-7B. Why are the corresponding results on Qwen2.5-VL-7B omitted, and does the method remain effective on these benchmarks across model families?
-
HCA-Edit introduces several hyperparameters, including , , , and , potentially creating a substantial tuning burden in practical deployment. It would be helpful to provide a practical method for selecting hyperparameters.
-
Additional references on Attention Intervention to consider:
[1] Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens. Jiang Z, Chen J, Zhu B, et al. CVPR 2025.