- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a mechanistic analysis on the information flow of hidden representations of question tokens and how this information affects the model's prediction. The authors use activation patching as the main approach to compre the effects of replacing question token hidden states with those from different images (sharing or differing in answers) to the same question. The experiment spans multiple multimodal LLMs by constructing 538 quartets from VQAv2 and found that different-answer donors likely to drive the recipient toward the donor's answer whereas same-answer donors have less impact on output. Additionally, the authors found that the question-token representations can be decomposed into a low-rank subspace that captures part of the patching effect. Authors demonstrate that maniuplating this subspace representation can also have causal impact on model's prediction.
Critical strengths
-
The experimental design in this paper is sensible. Though I have some reservation of the problem scope (see weakness), I believe the experiments presented in this paper reasonably support the conclusion. Specifically, the paper intoduces a controlled activation patching setup while keeping the question fixed and matching the hidden norms, which helps disentangle the effects of the answer-related information (the main study subject in this paper) and other variations.
-
In addition, the experiment covers a wide range of multimodal architectures and different protocols such as layer-wise patching, low-rank subspace interventions, which make the experiment more compact.
Critical weaknesses
-
The problem scope of this paper and research insight is incremental to me. The main conclusion of this paper can be summurized as, the hidden states of question tokens can carry answer-related information in the middle layers and subsequent layers in the LLM use this information to make prediction. Although this exact conclusion is not stated by existing work in literature, a number of existing work studies the information flow of MLLM and some of them even studied this in a more generalized setup [1, 2, 3]. [1] and [3] already studied the information flow within MLLM and [3] even shares some conclusion of this paper that the visual information can be mediated through question tokens to affect final predictions of MLLMs. The results related to middle layer integration are also presented by existing work [2].
-
The conclusion in this paper is well-expected given the current literature in community. It is not clear to me what additional insights I learned from this paper. In my current understanding, this study mainly focuses on the question token hidden states. But what additional understanding do we obtain by speficially studying the information flow of question tokens? I don't think there's enough justification on why question tokens are an important subject study or what broader impact this observation can make (e.g, improving model performance).
-
Even for this specific problem of information flow through question tokens, I think the study is not significant enough to publish at top-tier conferences. For example, the paper conducts a layer-wise study on patch layer yet the patch layer in other experiments do not make use of the result. Specifically, the layer sweep shows a peak of patching at 13 and 14 for Gemma3 and Idefics3 but the patch layer in other experiments such as low-rank subspace are still other predefined patch layers. Additionally, the experiment results show that different MLLM have different patching effect yet the paper doesn't investigate much into these aspects.
-
This is a minor point but the writing in this paper can be improved. While individual sections are written clearly, the transition and motivation of different studies are very weak. In the current form, it read like "We test A and A holds; we test B and B holds" but why these studies matters is not clearly justified in the paper. This is also related to the very motivation of paper: why studying question tokens is important?
[1] Basu et al, Understanding Information Storage and Transfer in Multi-modal Large Language Models [2] Yu et al, How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding [3] Salazar et al, Pathways of Visual Information Flow in Vision-Language Models
Questions for authors
-
Given the existing literature, could authors clarify on the significance of this study in terms of both the new insights this study establishes and the potential broader impact?
-
More related to the specific problem, could authors clarify why the information flow of question tokens is an important study subject?
-
As discussed in weakness, the results of the layer sweep are not made use in other studies such as the investigation of the low-rank subspaces. How would the results for low-rank subspace change if we use the peak-effect layers to learn the projection? Or more broadly, would there be a more holistc landscape for the low-rank subspace across different layers?
-
Could the authors also clarify how the pre-specified intervention layers were originally selected?
-
Do authors have any insights or hypothesis regarding the difference of patching effects across different MLLM families? Ideally, it would be helpful if these hypothesis can be backed by empirical evidence.
6 comments
Response to Reviewer n3Vn (2/3)
W3. The layer-sweep results are not used in the other studies: the sweep peaks at layers 13 and 14 for Gemma3 and Idefics3, yet the low-rank analyses use other, predefined layers. The paper also does not investigate why patching effects differ across MLLMs.
Q3. How would the low-rank results change if the projection were learned at the peak-effect layers, and is there a more holistic landscape across layers?
A3: Thank you for this suggestion. We find that the component becomes much stronger when fitted inside the window, and that its different uses peak at different depths. We added these results as a new Section 5.5, with details in the new Appendix H.5:
- Idefics3. We refit the component at the sweep-selected layer 14 on the same 320 evaluation pairs. The share of the donor effect that it carries rises from 51% to 92%, and its retention and removal specificity rise from 0.28 to 1.14 and from 0.36 to 1.52. The result holds across three initialization seeds.
- Gemma3. On a separate panel of 270 mostly yes/no quartets, the projected-source effect is 0.20 at layer 13 but only 0.015 at layer 21.
- Qwen2.5-VL-7B across layers. Retention specificity is 0.01, 0.48, and 0.47 at layers 13, 17, and 21, whereas the shared direction flips 91.6% of yes/no decisions at layer 17 but only 1.6% at layer 21.
We also tested the sweep-selected layers on new data. On 1,000 new GQA quartets, which we used neither to select the interfaces nor to locate the windows, the contrast at the sweep-selected layer is 5.3 times that at the fixed layer in Idefics3 and about 20 times in Gemma3. We added this to the paragraph "The summary resides in a middle-layer window" in Section 4.
Therefore, the component is much stronger inside the window, and answer transfer and steering should be placed at different depths.
Q4: Clarify how the pre-specified intervention layers were originally selected.
A4: Thank you for this question. We selected the Qwen2.5-VL-7B layer in an earlier pilot and transferred its relative depth to the other models before evaluating them. We added this history to the paragraph "Patching question-token states" in Section 3 and as a new paragraph in Appendix A.3:
- Qwen2.5-VL-7B. The pilot used 120 GQA examples and screened eight depths. It scored each depth by how well clean question-token and visual-token states restore an image-corrupted answer in all three task families, and it selected layer 17.
- Other models. Qwen2.5-VL-3B used the same candidate position, layer 22. We mapped the other five checkpoints to the same normalized depth, which gives layers 20, 21, 22, 22, and 20 for Idefics3, Gemma3, Qwen3-VL-8B, InternVL3.5-8B, and Phi-3.5-vision. We fixed these layers before evaluating each model.
This history also supports one of our recommendations. The windows lie at about 0.63, 0.45, and 0.39 of the network depth in Qwen2.5-VL-7B, Idefics3, and Gemma3, so a relative depth transferred from one model can miss the window of another. We therefore recommend sweeping layers before any fixed-layer analysis.
Q5: Provide insights or hypotheses, ideally backed by empirical evidence, about why patching effects differ across MLLM families.
A5: Thank you for this question, which also covers the second part of W3. We find that model differences come from two sources: where the window lies and how each model routes the answer. We added this as a new paragraph, "Why models differ", in the new Discussion section, Section 6:
- Window depth. Moving into the window raises the share that the Idefics3 component carries from 51% to 92%, raises the projected-source effect of Gemma3 about fourteenfold, and enlarges the GQA contrast of both models, as described under Q3.
- Routing. Inside the window, Qwen2.5-VL-7B concentrates the answer contrast in its question tokens, whereas Idefics3 carries as much in its image tokens, as described under Q2.
Therefore, models differ mainly in where and through which tokens they carry the answer summary, not in whether they carry it.
Response to Reviewer n3Vn (3/3)
W4: The writing can be improved, and why studying question tokens matters is not justified.
A6: Thank you for this comment. We revised the paper so that each study follows from the previous one. Specifically, we made three changes:
- We added two sentences to the first paragraph of the introduction that explain why the central question matters: if later layers read only an answer-level summary, the question tokens are where image evidence becomes a decision, and the answer decides what patching these states measures.
- We restructured the rest of the introduction as four questions, each motivated by what the previous one leaves open: whether a swap acts through the answer, where the summary resides, which part of the state carries it, and what it encodes.
- We rewrote the opening and "Summary" paragraphs of Section 4 to state the competing accounts each analysis tests and what remains open for Section 5, and we added a Discussion section, Section 6, that states what the results imply.
Therefore, each experiment now answers a question raised by the previous one instead of standing alone.
Thank you again for your feedback. We hope our responses address your concerns, and we would be grateful if you would consider updating your evaluation.
Best, Authors
Response to Reviewer n3Vn (1/3)
Dear Reviewer n3Vn,
Thank you for your detailed review, for finding our experimental design sensible and our experiments reasonably supportive of our conclusions, and for noting that the fixed-question, norm-matched setup disentangles answer-related information across a wide range of architectures and intervention protocols. In the revised PDF, we mark changes addressing your review in blue and changes addressing concerns shared with other reviewers in purple. We refer to your weaknesses as W1–W4 and your questions as Q1–Q5, in the order of your review, summarize each point before our response, and tag each change in the PDF with the same numbers.
W1. The problem scope and research insight are incremental, because existing work already studies information flow in MLLMs, including visual information mediated by question tokens and middle-layer integration.
W2. The conclusion is well expected, and the paper does not justify its additional insight, why question tokens are an important subject, or its broader impact, such as improving model performance.
Q1. Clarify the significance of the study, in terms of both the new insights it establishes and its potential broader impact.
A1: Thank you for pointing us to these works. Our main point is that prior work shows where visual information travels, whereas our design reveals what the question tokens relay. Basu et al., Yu et al., and Salazar et al. trace storage and transfer, processing stages, and pathways, but none of them holds the answer fixed while changing the image. Our same-answer donor adds this control, and it yields four findings that these designs cannot establish:
These findings also have broader impact: the component can control generated answers, the best intervention layer depends on the type of intervention, and patching studies of VLMs need a same-answer control. We added these points as a new "Implications" paragraph in Section 6. We also added Yu et al. to the introduction and to the related work in Section 2, and revised both to state this distinction.
Therefore, our study establishes what the question-token pathway carries and how the model uses it, which the routing analyses of prior work leave open.
Q2: Clarify why the information flow of question tokens is an important subject of study.
A2: Thank you for this question, which also covers the question-token part of W2. We study the question tokens for two reasons, and we added both to the first and third paragraphs of the introduction. First, later layers rely on these states: swapping only the question-token states makes 78% of the answers of Qwen2.5-VL-7B follow the donor, even though the recipient keeps its own image tokens, as Section 5.4 of the original submission reports. These states are therefore where image evidence becomes a decision. Second, their text is identical across images, so their states differ only in what they read from the image, which makes them a controlled site for our comparison.
We also tested whether the answer contrast resides in these tokens, by applying the same swaps to the image tokens and the final prompt token. In Qwen2.5-VL-7B at layer 17, the contrast is 1.22 for question tokens, but only 0.004 for image tokens and essentially zero for the final prompt token. In Idefics3 at layer 14, image tokens carry a contrast as large as question tokens, which marks a genuine difference in how the two models route the answer. We added this test as a new paragraph in Section 4, "Which token positions carry the contrast?", with details in the new Appendix H.1.
Therefore, the question tokens are both the cleanest site for an answer-controlled test and, in Qwen2.5-VL-7B, the site where the answer contrast is concentrated.