acceptodds
Under review as a conference paper at ICLR 2027

What Keeps Vision-Language Models Looking at the Image?

Abstract

When do vision-language models need direct access to the image while generating an answer? We study image dependence during answer generation by examining how the visual information needed for the current question becomes available in context. We intervene on direct image access while retaining previously computed states. Across real-image and synthetic tasks, we show that, depending on the generation process, direct access can continue to support accuracy after question processing. Supplying the required attributes as text in the context weakens this dependence. On synthetic tasks, we also examine how dependence changes as the model itself states the required attributes. Before attribute expression, severing access reduces accuracy, and replacing image-side states shifts answers toward the counterfactual content. After sufficient expression, both interventions have smaller effects. Even with an identical generated prefix, dependence differs according to whether image access was available during question processing. Thus, both the visible text and the preceding image access matter. Several of these patterns hold across model families, including Qwen2.5-VL-32B and InternVL3-14B. These findings offer a view of image dependence in terms of the information needed for the current question and the history of image access, beyond generation position alone. This perspective provides a basis for deciding when to reduce visual access during an answer and which visual information to retain for subsequent questions.

Primary area:
interpretability and explainable AI
Keywords:
Vision-Language Models, Multimodal Reasoning, Chain-of-Thought, Interpretability, Attention Interventions

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_T2mtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates when vision-language models (VLMs) still need to access an image while generating answers and how this dependence changes as relevant visual information becomes available. The authors use attention knockout to block direct access to image tokens at different stages of processing, while preserving previously computed internal states. The paper suggests that both previously acquired visual information and the content of generated reasoning influence when direct image access is needed.

Critical strengths

The paper addresses an interesting and important question about how VLMs use visual information during reasoning. I appreciate its careful experimental design, which uses several complementary interventions rather than relying only on attention scores. I particularly liked the finding that even with identical generated text, the model's dependence on direct image access can differ depending on whether it had access to the image earlier.

Critical weaknesses

My main concern is whether the experiments provide a sufficiently clear explanation of how visual information is retained and used inside the model. Attention knockout blocks direct access to image tokens, but it does not remove visual information already stored in the previously computed hidden states. Therefore, a model that continues to answer correctly after knockout may still be relying on visual information acquired earlier. Although the authors acknowledge this limitation, the experiments do not directly identify what information is retained in these internal states or how it contributes to the answer.

Questions for authors

How can you distinguish whether reduced image dependence is caused by the model expressing the required visual information in text, or simply by the model having already processed enough information to determine its answer?

The attention-knockout experiments preserve previously computed hidden states, which may already contain substantial visual information. Have the authors considered using activation patching or contrastive hidden-state analysis to investigate what visual information is retained in these states? For example, comparing and selectively replacing question-token representations obtained with and without image access could help identify whether the model has already encoded the required visual attributes or computed the answer before generation. Such an analysis might provide a stronger mechanistic explanation for why direct image access becomes less important at certain stages.

Reviewer_cM9POpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when do VLMs need direct access to the image while generating an answer. I first thought this paper would be about benchmarks and priors, but it is more about the answer generation phase. An attention knockout method is used and the dependency on images is measured by the accuracy loss.

Critical strengths
  1. The presentation is clear and the results are easy to understand.

  2. The method of measuring the dependency of images seems solid.

Critical weaknesses
  1. The results look a bit trivial to me. Two major results are listed by the authors, first "direct image access can continue to support accuracy during CoT generation, whereas direct answers are robust", second "supplying the required attributes in text reduces this dependence". I think these two results are both expected and do not really provide insights for the community.

  2. It seems like only the knockout loss is reported and the absolute accuracy is missing. I think the results would be weaker if the attention knockout only makes the performance of CoT degenerate to direct answer, since that is expected? I did not read the appendix, so perhaps I missed it.

  3. The choice of models are rather old. The main model Qwen2.5-VL is actually well-known to be imperfect now. But I understand the resource constraint.

Questions for authors

Do you have the results of absolute accuracy? I am curious to see the comparison between direct answer and CoT after the knockout.

Reviewer_cfEoOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Understanding when the model needs direct image access is important for model applications. This paper studies image dependence in vision-language models by examining how the visual information needed becomes available in the context. To study this problem, this paper designs a framework that operationalizes image dependence as knockout sensitivity. Based on this framework, this paper further analyzes the influence of different factors (e.g., response format) on image dependence and reveals several insightful findings.

Critical strengths
  • This paper studies direct image dependence in vision-language model, which is an important topic for applications, such as from efficiency perspective.
  • This paper performs a systematic analysis on both synthetic and real datasets. It also reveals multiple insightful findings, such as how response format change the image dependence.
  • This paper is well-written and easy to follow.
Critical weaknesses
  • Task coverage from the analyzed dataset.

As discussed in Section 2.3, this paper selects specific tasks from the analyzed dataset, such as the relation-color questions from GQA. Why is this specific task selected? Could the findings in this paper generalize to other task types?

  • Discussions about why direct answers and CoT answers perform differently.

Section 3.1 reveals the difference of image dependence between direct answer and CoT setting. Is there further discussion about why this happens? For example, does the token length in CoT setting influence the results?

  • Discussion about how to improve the models further.

This paper reveals multiple findings regarding how different factors change the image dependence in vision-language models. One question is: could we improve models based on these findings?

Questions for authors

Please refer to the weaknesses.

Comments on the paper

1 comment
Area_Chair_LEr1OpenReview ↗

Discussion phase will end on Nov 18

Dear Reviewers,

This is a kind reminder that discussion phase will end on Nov. 18. As the authors have provided the rebuttal, could you please check authors' rebuttal and update your thoughts? Your early feedback would make the discussion more smooth and informative.

Thank you.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.