- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
VLMs can be distracted by the textual context that conflicts with the image and generate wrong response, even though the model can generate the correct answer when it is not given the context. Therefore, the authors propose P-GVAR, a training-free method, which selectively uses the response obtained with the input context removed. P-GVAR decodes several responses for different input conditions, and considers the context as not useful when the response with context and the response without context are different, and the response given the context with the image and without the image are the same.
Critical strengths
None
Critical weaknesses
1. The authors’ suggested contributions are not supported by their experiments, or some of them are well-known general knowledge that prior works have dealt with.
Observation in Section 3: Compared to the VLM’s performance in the context-free setting, higher performance with matching context and lower performance with corrupted context is how the evaluated benchmark (Words-or-Vision[1]) was already designed, and it is a generally known fact [2, 3].
Also P-GVAR selectively withholds only the unhelpful context, but the usefulness of this candidate selection is rather refuted by their own experiments. In Table 1, the proposed P-GVAR performs worse than not showing the context at all (B-only). Even in Table 4, where only helpful context is given (matching), P-GVAR performs similarly to B-only, and this contradicts the authors’ claim.
Also, comparisons with directly related works are lacking. Among the compared methods, none other than RVD deals with the situation where mismatched context is given. Prior training-free methods which deal with modality conflict check if the image and the context are matched by prompting [2], or more directly and almost similar to P-GVAR, solve the task using image input, then solve the task using the text input, and then inspect whether any inconsistencies between the outputs exist [3].
[1] Deng et al. Words or Vision: Do Vision-Language Models Have Blind Faith in Text?. CVPR 2025
[2] Zhang et al. Robust Multimodal Large Language Models Against Modality Conflict. ICML 2025
[3] Pezeshkpour et al. Mixed Signals: Decoding VLMs’ Reasoning and Underlying Bias in Vision-Language Conflict. EMNLP Findings 2025
2. The paper makes a precise understanding of the method and reproducibility impossible.
There is no concrete description or example of the prompts used. There is no explanation of the visual-applicability check used in the method. There is no explanation of how response validity is measured. There is no description of the main benchmarks. Also the authors use unintuitive and inconsistent terminology (visual reset: just the ordinary VQA setting without external context, re-grounding: this also seems to be just the ordinary VQA setting). The table captions do not follow the placement rule. And there are typos and unclear sentences, symbols used without definition or explanation, some contents described without referencing the related figure, and a lack of explanation for some results (What are Refresh, Recovered, Harmful, and Invalid in Table2? What is Rule in Table 1 and Table 7?)
Questions for authors
See the weakness.