acceptodds
Under review as a conference paper at ICLR 2027

Artemis-3D: Latent Spatial Reasoning for Visual-Spatial Intelligence

11 views

Abstract

Visual-spatial intelligence poses significant challenges for current multimodal large language models (MLLMs), whose language-centric representations limit their ability to reason about spatial relationships. Existing approaches address this limitation by either injecting geometric information through external modules or verbalizing spatial information into textual Chain-of-Thought (CoT), yet both ultimately retain a language-oriented reasoning space. Meanwhile, we observe that language-oriented visual representations can conflate semantically similar objects at distinct spatial locations, obscuring the spatial distinctions essential for reasoning. To overcome the constraints of language-oriented spatial reasoning, we introduce Latent Spatial Reasoning, a new paradigm that enables MLLMs to construct and reason directly within a spatially grounded latent space. We instantiate this paradigm in Artemis-3D, a unified framework with two complementary components. First, Mental Map Imaging (MMI) internalizes geometric structure into visual token representations, constructing a latent mental map that preserves spatial distinctions. Second, Visual-Cue Latent Reasoning (VCLR) autoregressively retrieves question-relevant spatial evidence from this mental map and performs variable-length reasoning directly in the spatial latent space. Together, MMI and VCLR enable Artemis-3D to preserve geometric structure throughout the reasoning process without explicit textual CoT. Extensive experiments demonstrate that Artemis-3D outperforms state-of-the-art methods on visual-spatial intelligence benchmarks. Our code will be publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Spatial Reasoning, Latent Reasoning, Geometry-Aware Visual Representations, Mental Maps

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_QANzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Artemis-3D, a framework for latent spatial reasoning that enables MLLMs to construct internal geometric representations and reason directly in the latent space without explicit textual intermediate steps. It consists of two components: Mental Map Imaging (MMI), which uses geometric supervision to encode 3D scene structure into visual hidden states, forming an internal mental map, and Visual-Cue Latent Reasoning (VCLR), which autoregressively generates spatial latent tokens to retrieve question-relevant geometric evidence from this map. At each reasoning step, the predicted latent token retrieves the most similar geometric token, which is fed back to guide subsequent reasoning until a progress head signals completion and the model generates a textual answer. The framework requires no external geometry models at inference and is evaluated on VSI-Bench, MindCube-Tiny, and Real-3DQA under task-specific training settings.

Critical strengths
  1. A compelling framework for latent spatial reasoning. The paper addresses the challenge of preserving scene geometry and leveraging it during intermediate reasoning. MMI incorporates geometric supervision into visual representations, while VCLR enables sequential reasoning over question-relevant spatial cues. Together, they provide a coherent framework without requiring external geometry models at inference.

  2. Strong empirical results with supporting ablations. The 2B model achieves 72.5 on VSI-Bench. Same-backbone ablations show that MMI and VCLR each contribute to performance improvements, with their combination yielding a 4.5-point gain over the SFT baseline. The method also achieves competitive results on MindCube-Tiny and Real-3DQA under task-specific training settings.

Critical weaknesses

The paper tackles an important spatial reasoning challenge with a technically comprehensive framework and promising empirical results. However, several concerns remain:

  1. The contribution of iterative cue retrieval remains unclear. Tables 3 and 4 show improvements from VCLR, but do not isolate whether the gains arise from iterative geometric cue retrieval and feedback rather than additional supervision or inference steps. A controlled comparison removing or perturbing retrieval while preserving latent reasoning steps would better validate the proposed mechanism.

  2. Limited controlled comparisons with alternative approaches. Artemis-3D aims to internalize geometry without external 3D encoders, but existing comparisons differ in training data and input settings. Matched comparisons with encoder-based or textual reasoning baselines, including inference costs, would better establish the benefits and accuracy–efficiency trade-off of the proposed latent formulation.

Questions for authors
  1. Visual-cue ordering and cross-frame consistency. How is the target visual-cue sequence ordered? When an object appears across multiple frames, are its tokens treated independently, and could cross-view feature variations affect retrieval stability?

  2. Verification of reported results. The eight category scores in Table 3, row (0), average to approximately 64.6, rather than the reported 65.2. Figure 10’s caption also describes an object-counting failure, while the figure shows a viewpoint question. Some comparison settings could be further clarified to ensure fair evaluation. For example, in Table 1, GeoThinker’s 50.5 appears to correspond to the Qwen2.5-VL-7B, 8-frame setting in the cited paper.

2 comments
Reviewer_QANzOpenReview ↗

Reply

Good work, and thanks for the response. I have no further questions.

We are glad that our response has addressed your questions. We sincerely appreciate your thoughtful review and positive feedback.

Reviewer_1JyuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that spatial reasoning in MLLMs is constrained by a language-oriented representation space: a de-tokenization + UMAP analysis (Fig. 1) shows that an MLLM's visual tokens cluster by semantic category even for spatially distinct instances, whereas a geometric foundation model (VGGT) separates them. The authors propose "Latent Spatial Reasoning" and instantiate it as Artemis-3D with two components. Mental Map Imaging (MMI) supervises intermediate-layer visual tokens (plus learnable camera/scene/scale tokens) with depth, point-map, camera-pose, and metric-scale reconstruction objectives using pseudo-labels from frozen geometric foundation models, producing an internal "geometric token bank" (mental map). Visual-Cue Latent Reasoning (VCLR) then performs autoregressive next-latent-embedding prediction over this bank: a trace of question-relevant visual-cue tokens is constructed from automatically annotated bounding boxes, trained with a cosine-similarity objective, stabilized by a gated latent adapter with step embeddings, and terminated adaptively by a progress head. At inference, predicted latent tokens are greedily matched (no-repeat) to the bank and fed back.

Critical strengths
  1. A clearly articulated and well-evidenced motivating observation. The de-tokenization/UMAP analysis showing that MLLM visual tokens conflate semantically similar but spatially distinct objects—while a geometric model separates them—is a simple, convincing representation-level diagnostic that grounds the paper's premise better than the usual behavioral-only arguments.
  2. A coherent instantiation of a genuinely interesting paradigm. Doing reasoning as autoregressive retrieval over an internal geometric token bank (with a progress head for variable-length termination and traces spanning 20–200+ steps) is a distinctive design that goes beyond prior visual latent reasoning (CoVT, LVR, LIVR, Monet), which uses fixed-length or reconstruction-oriented latents. The "reason in the space where the geometry lives" thesis is appealing and the authors commit to it fully.
  3. Reasonable ablation coverage. The MMI/VCLR synergy ablation (+4.5% jointly vs. +2.0/+0.8 individually), component-wise ablations of the adapter/trace/progress head, decoding strategies, and the ScanNet reconstruction check (Table 4d) showing the internal map retains decodable geometry are all useful evidence that the components behave as intended.
Critical weaknesses
  1. The visual-cue trace supervision depends on an unspecified annotation pipeline. Sec. 3.2.1 states that "an automatic annotation pipeline" extracts question-relevant bounding boxes, but neither the main text nor the appendix says what produces these boxes (a detector? an MLLM? GPT-based parsing of question nouns?), how noisy they are, or what happens for questions without clearly localizable regions. Since the entire VCLR supervision signal (trace order, region coverage, progress-head labels) derives from these boxes, this is a significant reproducibility and validity gap.
  2. Headline margins over the strongest baselines are thin. VSI-Bench: 72.5 vs. SpatioLM-8B 71.6 (+0.9) and Cambrian-P-7B 72.0 (+0.5); MindCube-Tiny: 87.2 vs. SpatialDreamer 84.9 (+2.3). No variance or significance reporting is provided, and VSI-Bench numbers are known to be noisy across evaluation harnesses. The "outperforms SOTA" claim rests on sub-1% margins.
  3. A notable deficit on the hardest spatial category is not discussed. On MindCube-Tiny, Artemis-3D scores 69.5 on Rotation vs. SpatialDreamer's 87.5—a 18-point gap on the most viewpoint-transformation-centric split—while the overall win is driven by Among (93.6) and Around (86.0). This cuts against the paper's central claim and deserves analysis.
  4. No controlled comparison against textual CoT on equal footing. The paper's motivating argument is that verbalizing spatial information into CoT is inferior to latent reasoning, but all CoT-based comparisons are external published systems with different training data and pipelines. An in-house textual-CoT baseline trained on the same VSI-590K/VLM-3R data (e.g., verbalized camera/BEV traces à la Map2Thought) would isolate the contribution of the latent space itself. As Table 3 shows plain SFT already lifts Qwen3-VL-2B from 50.3 to 65.2, the marginal value of the full latent machinery (+4.5) needs this control to be convincing.
Questions for authors

See weaknesses above

Reviewer_WzXjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Artemis-3D aims to improve spatial reasoning in multimodal language models through geometry-grounded latent representations. it combines Mental Map Imaging (MMI), which uses geometric reconstruction supervision to encode scene structure into visual hidden states, with Visual-Cue Latent Reasoning (VCLR), which autoregressive retrieves relevant geometric tokens before generating an answer. A progress head enables variable-length reasoning. The method achieves competitive results on VSI-Bench, MindCube-Tiny, and Real-3DQA.

Critical strengths
  1. Coherent framework: MMI and VCLR connect geometric representation learning with subsequent evidence retrieval, providing a concrete mechanism for exploiting internal spatial representations.
  2. Competitive small-model performance: The 2B model achieves 72.5 on VSI-Bench and 87.2% accuracy on MindCube-Tiny, outperforming several larger models.
  3. Supportive ablations: combining MMI and VCLR improves the SFT baseline from 65.2 to 69.7, exceeding the gains from either component alone.
  4. Useful representation analysis: Geometry reconstruction and visual-cue visualizations provide evidence that the learned representations retain spatial information.
Critical weaknesses
  1. The spatial reasoning claim is insufficiently supported. VCLR learns to retrieve relevant visual tokens, but the experiments do not distinguish spatial reasoning from sequential evidence aggregation. Stronger controls, such as shuffled traces and compute-matched aggregation baselines, are needed.
  2. Viewpoint robustness remains limited. Artemis-3D trails GPT4Scene-HDM on Real-3DQA’s VRS (13.5 vs. 18.2) and SpatialDreamer on MindCube-Tiny’s Rotation subset (69.5% vs. 87.5%), weakening the claim of improved viewpoint reasoning.
  3. The source of the gains is unclear. VCLR introduces both additional region supervision and inference steps, but their contributions are not isolated. Missing latency and accuracy–compute comparisons further limit assessment of its practical benefits.
Questions for authors

See Critical Weaknesses

Reviewer_SHHqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Artemis-3D, a framework for spatial reasoning through geometrically grounded latent representations. Mental Map Imaging (MMI) supervises internal visual states with depth, point-map, camera, and scale reconstruction objectives. Visual-Cue Latent Reasoning (VCLR) then autoregressively retrieves geometric tokens and feeds them back into the model before answer generation. Experiments with Qwen3-VL backbones report strong performance on VSI-Bench, MindCube-Tiny, and Real-3DQA.

Critical strengths
  • The paper is well organized, and the combination of geometry-supervised representation learning and visual-cue retrieval forms a clear and reasonable pipeline.
  • The experiments report strong QA performance with small backbones. Component ablations and quantitative geometric reconstruction results provide useful support for the proposed design.
Critical weaknesses
  • Unclear justification of VCLR. VCLR uses coverage of selected cue patches as its completion target, but patch coverage does not necessarily indicate completed spatial reasoning. It remains unclear whether sequential traversal offers benefits beyond simply retrieving relevant evidence.

  • Insufficient support for spatial reasoning claims. Table 3 shows a route-planning regression from 48.4 to 44.9, while Table 2b reports weaker viewpoint-rotation consistency than GPT4Scene-HDM (13.5 versus 18.2). These results require further analysis to support the claimed spatial reasoning benefits.

Questions for authors
  1. Why is coverage of all selected cue patches used as the completion target? How does VCLR compare with selective top-k retrieval or shorter trajectories under matched settings?
  2. Can the authors test whether answers depend on the latent trajectory, and whether its ordering matters, using controlled replacement, masking, or shuffling?
  3. How do the authors explain the route-planning regression in Table 3 and the weaker viewpoint-rotation consistency in Table 2b? What failure modes account for these results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.