- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes S-Space, a three-dimensional linear subspace in the intermediate hidden representations of multimodal large language models (MLLMs). By computing gradients of spatial answer-logit contrasts with respect to object-token representations, the authors identify horizontal, vertical, and depth directions, enabling continuous spatial coordinates to be read out without additional training.
The authors validate S-Space across multiple MLLMs and spatial reasoning benchmarks, examining its coordinate accuracy, robustness across prompts, and causal influence on model predictions. They further explore its representations of unseen entities and abstract spatial concepts. Through explicit coordinate rotations, chain-of-thought (CoT) tracking, and reference-frame analysis, the paper distinguishes spatial perception from subsequent spatial computation. The results suggest that MLLMs can encode relatively accurate spatial information yet still struggle with viewpoint transformations and reference-frame reasoning.
Critical strengths
1. A unified and interpretable framework for analyzing spatial representations.
The paper uses a simple gradient-based method to identify spatial directions and applies the same S-Space representation to coordinate readout, causal intervention, cross-prompt analysis, and reasoning-process tracking. The causal intervention experiments are particularly convincing, as they compare targeted coordinate edits with random perturbations and examine their effects on both spatial and non-spatial predictions. The experiments are systematic and mutually supportive, providing evidence beyond simple correlations between internal representations and ground-truth positions.
2. Valuable insights into the mechanisms underlying spatial reasoning failures.
A key contribution is the distinction between spatial perception and spatial computation. On SpinBench, reading out S-Space coordinates and applying explicit viewpoint transformations substantially outperforms both direct prediction and CoT reasoning. Together with the analyses of reference-frame confusion and representation changes during reasoning, these results provide useful insights into the limitations of MLLM spatial reasoning. This mechanistic understanding is more informative than simply improving benchmark accuracy.
Critical weaknesses
1. Insufficient empirical comparison with closely related prior work.
Prior studies, particularly Kang et al. [1] and Gurnee et al. [2], have investigated linear spatial representations in object tokens and Jacobian-based internal workspaces, respectively. Although the paper cites and discusses these studies, the distinctions are primarily conceptual, without direct comparisons under matched experimental settings.
Consequently, it remains unclear whether the proposed gradient-based identification method provides meaningful advantages over existing approaches in readout accuracy, robustness, or causal effectiveness. This makes it difficult to assess the methodological novelty.
2. Some claims about internal spatial computation require stronger evidence.
The substantial gains from S-Space + Rotation rely on predefined viewpoint transformations. Moreover, SpinBench mainly involves discrete right-angle rotations, implemented through coordinate permutations and sign changes. These results do not necessarily establish that S-Space supports more general three-dimensional spatial computations.
Additionally, the CoT tracking experiments use controlled reasoning sequences containing correct spatial relations, which may not fully reflect the model's internal computations during autonomous reasoning. While these experiments demonstrate that spatial information can be extracted and used effectively, how the model itself performs spatial transformations remains less clear.
Questions for authors
1. Comparison with existing spatial representation methods.
Could the authors provide a direct comparison between S-Space and the spatial direction extraction method of Kang et al. [1], using the same models and datasets?
- Does the gradient-based method offer measurable advantages in coordinate readout, cross-prompt generalization, or SpinBench viewpoint transformations?
- How much does the query-reference gradient difference contribute compared with simply averaging gradients?
2. Generalization to more complex spatial transformations.
SpinBench mainly evaluates discrete right-angle viewpoint changes, where rotations can be implemented through coordinate permutations and sign changes.
- Is there evidence that S-Space supports more general, continuous viewpoint transformations?
- This would help establish whether S-Space captures a geometrically consistent three-dimensional representation rather than three independently readable spatial directions.
3. Spatial representation changes during autonomous reasoning.
Section 5.2 analyzes S-Space using controlled CoT sequences containing correct spatial relations.
- Do similar coordinate transformations emerge during the model's self-generated reasoning, particularly when comparing correct and incorrect reasoning trajectories?
- Such evidence would help clarify whether S-Space participates in the model's actual spatial computation process.
References
[1] Kang, R., Chen, H., Gkioxari, G., and Perona, P. Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models. International Conference on Learning Representations (ICLR), 2026. https://arxiv.org/abs/2601.12626
[2] Gurnee, W., Sofroniew, N., Pearce, A., Piotrowski, M., Kauvar, I., Chen, R., Soligo, A., Bogdan, P., Ong, E., Wang, R., Thompson, B., Abrahams, D., Kantamneni, S., Ameisen, E., Batson, J., and Lindsey, J. Verbalizable Representations Form a Global Workspace in Language Models. Transformer Circuits, 2026. https://transformer-circuits.pub/2026/workspace/index.html
4 comments
Part 2/2
(Continued from Part 1/2)
Q1b (Q1): Gradient difference vs. gradient averaging
In relative spatial queries, query and reference carry opposite spatial sensitivities while sharing the same image and prompt framing: and (Sec. 2.2 and Appendix B.1, Eq. 11). Consequently, role-averaging cancels the opposing spatial terms and isolates prompt drift, whereas single-token retains uncancelled prompt-context bias on every example. Taking the role-antisymmetric difference reinforces both spatial terms while cancelling shared prompt context, substantially improving readout, cross-prompt transfer, and SpinBench rotation (Appendix B.1, Tab. 2, Fig. 11):
| Method (Molmo2-ER) | Gradient estimator | CV-Bench | EmbSpatial | SpatialTunnel | Single-object / Non-spatial | SpinBench |
|---|---|---|---|---|---|---|
| S-Space | 91.9% | 86.9% | 96.5% | 87.3% / 88.6% | 94.5% | |
| Query token | 86.0% | 84.7% | 92.5% | 83.4% / 83.4% | 85.6% | |
| Role average | 30.2% | 26.7% | 15.3% | 33.2% / 30.4% | 33.6% |
Q2 (W2 / Q2): Continuous 3D viewpoint rotations
SpinBench evaluates discrete and viewpoint changes whose rotation matrices reduce to axis permutations and sign flips, which could still succeed even if and had incompatible scales. To test whether S-Space forms a geometrically consistent space that supports continuous viewpoint transformations, we evaluate clockwise observer yaw rotations across 200 real RGB-D test scenes from SUN-RGBD [5] using Qwen3.6-27B with frozen COCO axes and no scale calibration (Sec. 5.1 and Appendix C.5, Eq. 13, Tab. 19, Fig. 18).
Crucially, we read only once per scene from the front RGB image via standard left-right and closer-farther prompts, and then transform coordinates analytically for every target yaw angle via and , directly mixing and at intermediate angles. On the same 8,464 target-view queries, S-Space + Rotation substantially outperforms both Direct greedy decoding and Reasoning chain-of-thought generation while maintaining high accuracy on non-orthogonal views on par with orthogonal views:
| Viewpoint subset | Questions | Method | Left / right | Nearer / farther | Overall |
|---|---|---|---|---|---|
| Orthogonal, | 1,600 | Direct | 54.4% | 55.5% | 54.9% |
| Reasoning | 66.9% | 74.9% | 70.9% | ||
| S-Space + Rotation | 86.0% | 86.5% | 86.3% | ||
| Non-orthogonal, | 6,864 | Direct | 52.6% | 51.9% | 52.3% |
| Reasoning | 70.0% | 78.2% | 74.1% | ||
| S-Space + Rotation | 86.0% | 88.8% | 87.4% | ||
| All 24 angles (, step) | 8,464 | Direct | 53.0% | 52.6% | 52.8% |
| Reasoning | 69.4% | 77.6% | 73.5% | ||
| S-Space + Rotation | 86.0% | 88.4% | 87.2% |
[5] Song et al., SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite, CVPR 2015.
Q3 (W2 / Q3): Trajectories during autonomous reasoning
To examine whether similar coordinate updates emerge during the model's own spontaneous reasoning without controlled CoT templates, we evaluate Qwen3.6-27B on SpinBench given only the image and multiple-choice question with no CoT prefill (global response Q2.3; Sec. 5.2 and Appendix C.5, Tab. 21, Fig. 21). Across 8,505 complete self-generated <think> rollouts, we pair adjacent mentions of options A and B over normalized reasoning progress and fit the linear slope of the target-aligned coordinate difference.
Although free-form CoT is noisy rather than an exact geometric rotation at every step, rollouts reaching the correct final answer steadily shift object coordinates toward the target perspective (mean slope across 7,115 rollouts), whereas incorrect rollouts remain flat or drift away ( across 1,390 rollouts). Controlling for question difficulty across the 72 matched questions that contain both correct and incorrect rollouts yields the same contrast ( vs. , with higher mean slope for correct CoTs on 69 of 72 questions), confirming that S-Space actively participates in autonomous spatial reasoning.
Thank you for the detailed response and the additional experiments. The new results directly address my main concerns regarding methodological novelty, generalization to continuous viewpoint transformations, and spatial representations during autonomous reasoning. These additions substantially strengthen the paper's contributions and conclusions. Overall, I find the rebuttal convincing, and my concerns have been largely resolved. I appreciate the authors' efforts in addressing my questions.
Thank you for your thoughtful feedback and positive reassessment. We are pleased that our response and additional experiments have largely addressed your concerns. We would appreciate it if you could consider updating your rating to reflect your revised assessment.
Part 1/2
We sincerely thank Reviewer u2PB for the constructive review and insightful questions. We organize our response into Part 1/2 (
Q1abelow) and Part 2/2 (Q1b,Q2, andQ3), alongside the global response.Q1a (W1 / Q1): Comparisons with Spatial ID and Jacobian Lens
We benchmark S-Space against Kang et al.'s Spatial ID [1] and Gurnee et al.'s Jacobian Lens [2] on identical checkpoints, splits, and same-layer object-token hidden states (hq,hr) (global response Q1; Sec. 3.1-3.2, Appendix C.2-C.3). Because Spatial ID [1] relies on a synthetic 4x4 grid dataset and extracts only 2D axes without depth, we conduct a two-part readout comparison against it—first (Tab. 9) on 2D tasks and second (Tab. 10) adapting it to natural COCO images with depth labels—followed by causal swaps (Tab. 16, Fig. 13) and pretrained Jacobian Lenses [2] (Tab. 11):
What color is the {query}?, S-Space achieves higher counterfactual flips and higher color retention across α∈{1,…,5}:[1] Kang et al., ICLR 2026. [2] Gurnee et al., Transformer Circuits, 2026. [3] Yang et al., NeurIPS 2024. [4] Min et al., ECCV 2026.