acceptodds
Under review as a conference paper at ICLR 2027

Beyond Geometric Alignment: Making 3D Latents Matter for Spatial Reasoning

11 views

Abstract

Humans perform spatial reasoning by constructing internal spatial representations that support mental rotation, viewpoint transformation, and relational inference. Inspired by this ability, recent multimodal large language models introduce 3D latent reasoning, in which continuous latent tokens are expected to simulate spatial imagination steps. However, it remains unclear whether these latent tokens actually encode meaningful 3D geometry and contribute to spatial reasoning. In this work, we systematically analyze 3D latent reasoning and find that latent tokens have only weak causal influence on spatial reasoning and exhibit highly homogeneous representations across scenes. To reveal the underlying cause, we further identify an image-dominant shortcut during geometric alignment, whereby the alignment can be achieved largely from image features with limited reliance on 3D latents. Based on this diagnosis, we propose CALD-3D, a framework that combines contrastive geometric alignment with intra-latent diversity regularization to make latent tokens both geometrically meaningful and causally influential in spatial reasoning. Experiments on MindCube-Tiny and VSI-Bench demonstrate consistent improvements in spatial reasoning performance while substantially alleviating latent representation collapse.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Spatial Reasoning, Latent Visual Reasoning, Geometric Alignment, Contrastive Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_R1yaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates whether geometry-supervised 3D latent tokens effectively contribute to spatial reasoning. Through inference-time interventions and representation analyses, it finds that 3DThinker[1] exhibits weak dependence on latent tokens, limited scene-specific latent diversity, and an image-dominant shortcut in geometric alignment. To address these issues, the authors propose CALD-3D, which combines contrastive geometric alignment using cross-scene latent negatives with intra-latent diversity regularization to encourage more informative and distinct geometric representations. CALD-3D improves over 3DThinker on MindCube-Tiny (62.7% to 66.9%) and VSI-Bench (42.8% to 44.4%), with a notable 14-point gain on Rotation. Further ablations and latent interventions suggest stronger reliance on the latent pathway and reduced representation collapse.

[1] Chen et al. “Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views.”

Critical strengths
  1. A clear diagnosis of latent underutilization. Beyond showing that removing latent tokens has little effect on predictions, the paper separately perturbs image and latent inputs to the geometric projector, revealing an image-dominant shortcut in geometric alignment. This identifies a concrete limitation of 3DThinker [1] and provides a clear motivation for the proposed method.

  2. A targeted method with substantial Rotation improvements. CALD-3D contrasts matched and mismatched latents while keeping image features fixed, directly addressing the identified shortcut. It improves Rotation accuracy on MindCube-Tiny from 44.0% to 58.0%, with ablations and latent interventions further supporting the effectiveness of the proposed objectives.

[1] Chen et al. “Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views.”

Critical weaknesses
  1. The proposed mechanism is not fully verified. The paper shows that CALD-3D becomes more sensitive to latent interventions, but this does not establish that the latents encode scene geometry rather than other scene-specific cues. The image-versus-latent projector analysis is performed on 3DThinker, not repeated after CALD-3D training. Repeating that analysis, alongside a direct test of geometric content, would better support the central claim.

  2. The robustness and scope of the gains need clearer evidence. The MindCube-Tiny improvement is driven largely by Rotation (+14.0 points), while the VSI-Bench gain is modest (+1.6 points). Results from repeated training runs and CALD-3D latent interventions on VSI-Bench would help establish how consistently the findings extend beyond Rotation.

Questions for authors
  1. Does VGGT-based supervision encourage the latents to capture meaningful spatial structure, or primarily make them more distinguishable across scenes? A direct evaluation of their geometric content would help clarify this distinction.

  2. Is the 61.9% result in Table 4, obtained without the proposed losses, a reimplementation of 3DThinker under the same training data and settings as CALD-3D? How does it relate to the 62.7% result from the released 3DThinker checkpoint in Table 2?

  3. Are CALD-3D’s improvements consistent across training runs, and do latent interventions yield similar effects on VSI-Bench? Does the image-dominant shortcut identified in 3DThinker also occur in other geometry-supervised latent reasoning models?

Addressing these concerns would strengthen the paper and could raise my overall rating.

Reviewer_m53xOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether geometry-supervised latent tokens contribute to spatial reasoning. Through inference-time interventions, analysis of cross-sample diversity and sensitivity to scene changes, and separate interventions on the projector’s two input branches, it finds that the projector relies primarily on image features, with limited use of the latent branch. The proposed CALD-3D combines contrastive geometric grounding with intra-latent diversity regularization to improve the effectiveness of 3D latent reasoning.

Critical strengths
  • The diagnostic analysis provides useful insights that may inform subsequent research.
  • CALD-3D achieves performance improvements over 3DThinker.
Critical weaknesses
  • The identified image-dominant projector shortcut is established through analysis of 3DThinker, without validation on other latent reasoning models. However, the introduction and contribution statements do not clearly qualify this scope. Findings from a single model cannot be readily generalized to an entire class of methods.
  • The experimental comparison focuses on 3DThinker and lacks other representative latent reasoning baselines. The results also do not demonstrate a clear advantage over other types of models. Moreover, the experiments are limited to the Qwen2.5-VL-3B implementation of 3DThinker, making it difficult to assess the generality of both the diagnosis and the proposed method. The paper should clarify which conclusions are specific to this architecture (3DThinker) and which are intended to generalize to other geometry-supervised latent reasoning methods.
  • The ablations do not clearly demonstrate that CALD-3D mitigates the identified image-dominant projector shortcut. A direct analysis of the optimized model’s internal representations and branch contributions is needed to establish whether this mechanism has changed. The component ablations are also limited to MindCube-Tiny, without corresponding results on VSI-Bench.
Questions for authors

See weakness.

Reviewer_2QeEOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper finds that 3D latents often have little effect on spatial reasoning, proposes CALD-3D to make them more useful through contrastive alignment and diversity regularization, and reports improvements on MindCube-Tiny and VSI-Bench.

Critical strengths
  1. The strongest part is that the paper shows models can mostly ignore their 3D latents, then offers a targeted fix that improves results on two benchmarks.
Critical weaknesses
  1. The gains are pretty uneven. Rotation goes up 14 points on MindCube-Tiny, but Among barely moves, and the overall gain on VSI-Bench is just 1.6 points. I’m not sure how much to read into that from two benchmarks and one 3B model.

  2. Table 4 doesn’t quite tell me which loss is doing the work. It removes the diversity term first, then the contrastive term too, but never shows what happens when you remove only the contrastive term.

  3. The model clearly becomes more sensitive to the latent tokens. I’m less sure those tokens contain useful 3D geometry. Breaking a token sequence can hurt accuracy even if the model has learned to rely on something other than a good scene representation.

  4. As a method, this feels like a fairly small change to 3DThinker: the basic setup stays, with two losses added. I found the diagnosis more interesting than the proposed fix.

Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.