- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper discovers a shared semantic state within VLMs that explains diverse object hallucination patterns. The authors also show that this hidden state is distinct from the VLMs’ direct readout. These results demonstrate a shared hidden state within VLMs that routes diverse hallucination causes to diverse surface forms of the model answers.
Critical strengths
(1) The finding of a shared hidden semantic coordinate that explains different visual hallucination patterns is interesting.
(2) The control studies validating the presence and mechanistic interpretability of such an internal state are also thoroughly designed.
Critical weaknesses
(1) The paper cannot reliably claim "cross-model" signs based on just three specific small VLMs (Qwen3-VL-2B, Qwen2.5-VL-7B and InternVL2-7B), and these interpretations might not hold for larger VLMs. Additionally, apart from Fig. 2, the results in Tab. 2–3 are reported only for the 2B Qwen3-VL. Tab. 4 is also missing variants tested on InternVL. With most analyses done only on Qwen3-VL-2B, the conclusion doesn’t apply across models.
(2) Throughout the paper, only numeric statistics are reported. Since this paper is about a "semantic mediator" for visual hallucinations, one would expect to see qualitative examples demonstrating the effects. Moreover, the experiments are done on binary classification tasks only, where the reasoning of VLMs might be buried under the binary readout label.
Questions for authors
(1) Related to weakness (2), how does the interpretability finding transfer to, e.g., open-ended visual reasoning?
(2) The citation for CountBench is missing (Line 200).