- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies when VLMs turn visual information into spatial decisions. The authors use activation patching to trace how information moves from image tokens to option tokens and eventually to the final answer. Their main intervention reverses both the scene and the option order while keeping the correct answer unchanged. By comparing this intervention with scene-only and order-only changes, they identify a mid-depth commitment window where option representations become verdict-like and visual tokens lose their influence on the answer.
The authors find similar patterns across three VLM architectures and spatially fine-tuned variants. They also show that explicit reasoning can improve accuracy without making spatial predictions more stable.
Critical strengths
-
Well-designed counterfactual intervention. The answer-preserving reversal separates changes in visual content from changes in the correct answer. Together with the additional controls, it provides convincing evidence that option representations become verdict-like at intermediate layers.
-
Careful mechanistic analysis. The paper uses early-state restoration to test whether later layers stop using visual information or whether that information has disappeared. The results support reduced downstream use of visual tokens.
-
Broad experimental evaluation. The paper reproduces the main findings across different architectures and examines how spatial fine-tuning and explicit reasoning affect the process. The orientation results also show that visual handoff and commitment do not always coincide.
Critical weaknesses
-
The paper does not establish whether commitment explains spatial instability. The experiments show when option representations become verdict-like, but they do not show why spatial predictions become unstable. In particular, the authors do not test whether earlier commitment leads to greater instability or whether changing the commitment process makes predictions more robust. Without this connection, the results characterize how models form decisions rather than explain why those decisions are unstable. This distinction matters because spatial instability motivates the paper's central claim.
-
The contribution needs clearer positioning against prior work. Previous studies have shown that VLMs transfer visual information to text representations in early and intermediate layers and rely less on image tokens afterward (e.g., Kaduri et al., 2024; Basu et al., 2024; Zhang et al., 2025). This paper goes further by distinguishing visual content from the resulting decision. The authors should make this distinction central to their novelty claim and discuss the closest prior work more thoroughly.
-
The component analysis offers limited independent evidence for the commitment window. The authors report that 88–98% of influential components lie after commitment. However, knockout effects generally increase with depth, so many influential components would fall after any boundary near the middle of the network. The additional analysis in Appendix E.3 finds no distinct change in mediation at the commitment window after accounting for depth. These results are compatible with the authors' interpretation, but they do not provide independent evidence that commitment marks a distinct change in downstream computation.
-
The reasoning results need stronger statistical evidence. Table 4 shows modest improvements in accuracy alongside increases in prediction flips. However, the authors provide neither confidence intervals nor paired statistical tests for these differences. They should also report the number of eligible examples in the KV-cache experiments. These details are necessary to judge how reliably reasoning affects spatial stability.
-
The manuscript needs further editing and clarification. Table 2 reports 320 examples for InternVL2.5, compared with 446 for other models, without explaining the difference. Table 1 refers to three spatial relations, while Section 4.2 refers to four, and the paper does not define "Mirror error." The authors also report the direct-to-total ratio differently in the abstract (2–3%) and Section 4.3 (1.6–3.1%). Table 4's caption has a grammatical error ("is includes"). Appendix B contains an unfinished editorial note ("must be pasted from the released scripts before submission"), and the Reflection-V system prompt is missing. Several experimental details also need clarification. Several grammatical errors, awkward sentences, and inconsistent numerical formats (e.g., 0.848 vs .848) make the paper harder to read. A careful revision would improve both clarity and reproducibility.
Questions for authors
- For the answer-preserving donor, recovery is normalized by the image-only denominator even though the expected effect is zero. How many items have |LD_image − LD_ref| near the exclusion threshold, and does the 0.03 median change if normalized by the order-only denominator instead?
- Is the drop in R(O, L; both) at the window driven by option states no longer reading vision, or by the option states themselves changing representation? Patching only the vision→option attention edges at the window would separate these. Reflection-V describes the swapped scene in its trace but answers from the prior verdict. Is the final-answer preference at the end of generation still readable from the option-caption states, i.e., does the verdict-like state persist through the trace?
- On orientation, commitment lags handoff. Does recovery from the vision block show a second, later peak on these items, or is the gap filled by text-side computation alone?
- Please report the per-relation breakdown of the answer-preserving flip rates for the reasoning models; the aggregate hides whether the added instability is concentrated in orientation.