acceptodds
Under review as a conference paper at ICLR 2027

GeoLatent: Geometry-Guided Latent Structuring with Routed Optimization for 3D Reasoning

14 views

Abstract

Despite progress in vision-language models, 3D spatial reasoning from 2D images remains challenging. Text-based methods describe intermediate geometry with discrete tokens, limiting fidelity for continuous spatial relations. Continuous latents offer richer representations, but a single latent type does not explicitly separate the cues needed across spatial tasks. Decomposed spatial latents address this by representing position, direction, and global geometry separately under geometric supervision. Yet the geometry representation can still collapse toward one dominant direction, and unrestricted attention can leave the latents underused during answer learning. We introduce GeoLatent, combining Common–Residual Geometry Alignment (CR-GEO) with routed optimization to structure the geometry states while promoting latent-mediated answer learning. CR-GEO separates shared from residual teacher geometry; routed optimization jointly trains geometry and language, temporarily directs visual answer learning through the latents, and restores full attention with geometry supervision. In controlled comparisons, CR-GEO raises geometry effective rank from 1.00 to 3.87, while blocking latent readout at the bottleneck lowers direction accuracy from 89.1% to 25.8% on 128 fixed questions. After recovery, the differentiated geometry representation and latent-mediated visual route remain available alongside direct image access. GeoLatent achieves 73.0% on SPAR-Bench and 72.1% on SPBench, outperforming previously reported methods on both.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, 3D spatial reasoning, continuous latent reasoning, multimodal representation learning, geometric supervision, latent representation collapse

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iVT4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates 3D spatial reasoning for single-image VLMs and proposes GeoLatent. It uses CR-GEO to separate shared geometry from residual geometry, and applies routed optimization via joint–bottleneck–recovery to encourage the use of disentangled spatial latents. Experiments report an increased effective rank of GEO representations, strong latent dependency in the bottleneck stage, and 73.0% / 72.1% performance on SPAR-Bench and SPBench.

Critical strengths
  1. The diagnosis of GEO collapse is solid. Proposition 1 in Section 3.2 clearly analyzes why the original coverage loss fails to penalize collapsed configurations. More importantly, under bit-identical initialization and identical batch order, Table 2 and Fig. 3 only swap the GEO objective and raise reffr_{\mathrm{eff}} from 1.00 to 3.87. This structural difference is also reproducible on a second backbone, making CR-GEO more than an empirical tweak.

  2. The mechanism validation goes far beyond benchmark accuracy alone. The authors conduct route intervention, blank/mismatched-image tests, no-GEO controls, training overlap audits and cross-backbone reproduction. Tables 15–16 clearly distinguish enforced bottleneck dependency from post-recovery behavior. This restrained discussion of mechanisms improves overall credibility.

Critical weaknesses
  1. CR-GEO produces more diverse GEO features, but there is no proof these features genuinely help the final task. While the representation metrics change drastically, task accuracy gains are tiny, and most performance comes from POS/DIR. Current results only show GEO features become less redundant, and cannot confirm this diversity benefits spatial reasoning. The authors should run intervention experiments on existing checkpoints: keep images, POS/DIR and text fixed, only block or shuffle GEO readout and measure metric changes across sub-tasks. Run the same test for CR-GEO and the original L_cov baseline. If multiple random seeds are available, report mean and standard deviation for these small gains.

  2. The routed optimization only proves latent features are forced to be used in the training bottleneck and this latent path remains usable after training. It does not prove the model actually uses this path during normal full-attention inference. This idea draws inspiration from CVPR 2026 LIVR, so routed optimization is more an extension for geometry-supervised latent variables rather than a brand-new paradigm. The authors should test under standard full-attention decoding, keep direct image access enabled, and only block or shuffle latent K/V to measure performance drops against the control group. If performance barely drops, the claims should be limited: latent usage is enforced in training and the path stays available afterward, instead of claiming continuous latent-mediated reasoning at inference time.

Questions for authors

1.Do you have task-grouped GEO-only block or shuffle results for the recovered GeoLatent checkpoint? Specifically on SPAR Dep/Dis/View and ViewSpatial Camera, does CR-GEO’s GEO states yield measurable causal contributions compared with the original LcovL_{\mathrm{cov}}?

2.Beyond the 128 direction questions in Table 16, do you have results for blocking latent readout under normal full-attention inference across all three full benchmarks? Is the 0.78-point drop after recovery representative?

3.For the matched CR-GEO vs LcovL_{\mathrm{cov}} and bottleneck vs direct-recovery comparisons, do you have additional random seeds or training logs to verify whether these controlled gains of roughly 0.5–1.6 points are stable?

4.Given that this work explicitly builds upon LIVR’s visual bottlenecking (published at CVPR 2026), how do the authors define the substantive new additions of routed optimization relative to LIVR: the initial joint stage, typed/decomposed latent routing, ongoing geometry supervision, or a combination of all three?

Reviewer_k1QcOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies redundancy in geometric latent representations and the possibility that answer generation bypasses these latents during spatial reasoning in vision-language models. Building on GeoAnchor’s framework of interleaved text and POS, DIR, and GEO latent states, the authors propose Common–Residual Geometry Alignment (CR-GEO), which separately supervises the common geometric component and residual information in teacher features. The method also uses three training stages: joint training, visual bottleneck training, and recovery with full attention. The authors train Qwen3-VL-2B on 105,928 single-image spatial question-answering examples and achieve 73.0%, 72.1%, and 43.2% on SPAR-Bench, SPBench, and ViewSpatial, respectively. Validation includes loss replacement experiments with matched initialization, geometric representation diagnostics, local geometry decoding, attention interventions across training stages, and image replacement experiments. The results provide reasonably strong evidence for reduced representation redundancy and increased latent use during the bottleneck stage, but the independent contribution of the proposed components to final task performance remains insufficiently established.

Critical strengths
  1. The diagnosis of representation collapse is closely connected to the proposed method. Section 3.2 explains why the original coverage loss permits multiple GEO states to learn identical representations. The experiments with matched initialization in Figure 3 and Table 2(a) further show that replacing only the geometric objective increases effective rank from 1.00 to 3.87, accompanied by an approximately 1.3 percentage-point improvement in mean accuracy after joint training. This makes reducing GEO representation redundancy one of the paper’s better-supported contributions.
  2. The analysis of latent use is detailed and explicit about its scope. Figure 4(a) demonstrates strong latent dependence during the bottleneck stage, Table 3(b) supports the preservation of a usable visual pathway through the latents after recovery, and Figure 4(b) tests dependence on the specific input image. The authors distinguish these findings and explicitly acknowledge that latents do not necessarily remain the dominant information pathway after full attention is restored. Critical Weaknesses
Critical weaknesses
  1. The proposed components provide limited final performance gains, whose stability and generalization benefits remain insufficiently established. In Table 2(a), the mean accuracies at the end of recovery are 62.3% with the original objective and 62.8% with CR-GEO; §4.2 reports 62.4% for the no-GEO control. Thus, substantial changes in representation structure correspond to relatively small differences in final mean performance. In Table 1, GeoLatent improves over GeoAnchor on SPAR-Bench and SPBench, but ViewSpatial performance decreases from 47.0% to 43.2%, showing no consistent generalization advantage across benchmarks. Statistical tests on fixed checkpoints cannot replace independent training runs when assessing the stability of such small differences. In addition, Table 1 relies heavily on previously reported results, leaving comparisons against updated strong baselines under a common evaluation protocol insufficient.
  2. The paper does not sufficiently establish how the differentiated GEO content improves normal inference. Section 4.4 explicitly reports that, on the same set of 128 directional questions, blocking access to latent states reduces accuracy by 63.28 percentage points during the bottleneck stage, but by only 0.78 percentage points after full attention is restored. Table 3(b) establishes that a usable latent pathway remains after recovery, but this is different from showing that the specific geometric content learned through CR-GEO improves answers under normal full-attention inference. Because generation retains textual CoT as well as POS/DIR latents, the final gains remain difficult to attribute to GEO content rather than other intermediate representations or auxiliary supervision during training. This gap affects the assessment of the practical significance of the proposed geometric objective.
Questions for authors
  1. Evidence for the stability of final gains. For the approximately 0.4–0.5 percentage-point differences in final mean accuracy reported in Table 2(a) and §4.2, are results available across multiple random seeds under matched settings? Does the full method consistently outperform both the original GEO objective and the no-GEO control across independent training runs?
  2. Evidence for the role of GEO content. Under full attention after recovery, have you performed interventions targeting GEO content specifically—for example, replacing GEO states with those from another sample while preserving the number and positions of latent states, POS/DIR states, and attention structure? Can such results establish that the gains depend on sample-specific global geometric information, rather than only on the auxiliary training objective?
  3. Clarification of the interleaved reasoning design and its contribution. The interleaved text–latent structure in §3.1 is inherited from GeoAnchor. Beyond the existing motivation that text supports planning while latents represent geometry, is there evidence separating their actual contributions in this method? Under the same geometric supervision and comparable computational budgets, how does interleaved reasoning compare with removing intermediate text and generating a continuous sequence of latent states? The “Latent-only” routing experiment in Table 3 retains text and therefore does not answer this question.
  4. Clarification of baselines and comparison protocols. Could you include updated proprietary vision models and representative open vision models that were publicly available at the time of submission, evaluated on the same test subsets and with the same scoring protocol? Please report the exact model versions, image settings, and inference budgets. Please also clearly distinguish direct evaluation of general-purpose models from evaluation after training on spatial data from the same sources, so that gains from task adaptation can be distinguished from gains attributable to the proposed method.
Reviewer_qAinOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GeoLatent extends GeoAnchor with Common–Residual Geometry Alignment to reduce redundancy among geometry latents and a joint–bottleneck–recovery training schedule to encourage latent-mediated reasoning. Experiments show improved representation diversity and strong performance on SPAR-Bench and SPBench, supported by extensive representation and intervention analyses.

Critical strengths
  1. The method addresses clearly diagnosed failure modes with a motivated alignment objective and training strategy.
  2. Matched-initialization experiments and attention-route interventions provide substantial evidence beyond benchmark accuracy.
  3. The analysis carefully distinguishes representation structure, latent utilization, and visual dependence.
Critical weaknesses
  1. Limited scope beyond the GeoAnchor framework. The method largely inherits GeoAnchor’s architecture, latent decomposition, and geometric supervision setup, while the diagnosed collapse is closely tied to its coverage-based alignment objective. Although the proposed refinement is technically motivated, its broader applicability remains insufficiently demonstrated. Cross-backbone experiments retain the same framework and therefore do not resolve this concern. Comparisons with simpler anti-collapse objectives, such as latent decorrelation or teacher-feature centering, and validation in another latent-reasoning framework would help establish the necessity and broader significance of CR-GEO.

  2. Modest practical gains over controlled alternatives. The full method achieves a three-benchmark mean of 62.8, compared with 62.4 for the no-GEO control and 62.3 for the original-objective recovered control. An original-objective model with direct NTP continuation reaches 62.9 (Table 10). Thus, the substantial improvement in representation diversity does not consistently translate into superior task performance. Repeated-seed comparisons are needed to establish whether the small final gains are robust.

  3. Unclear contribution of structured latents during standard inference. On the direction-question subset, blocking latent readout reduces accuracy by 63.28 points during bottleneck training but only 0.78 points after recovery (Table 16). The bottleneck-stage dependence primarily involves POS/DIR, while blocking GEO alone has no effect on this subset. Although the route-retention experiments demonstrate an available latent pathway, they do not establish that the diversified GEO states materially contribute to final predictions under ordinary full attention.

Questions for authors
  1. How does CR-GEO compare with simpler anti-collapse objectives, and can its benefits be demonstrated beyond the GeoAnchor framework?
  2. Can GEO-specific interventions on appropriate tasks demonstrate that the diversified geometry states contribute to final predictions under standard full attention?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.