acceptodds
Under review as a conference paper at ICLR 2027

SpatialASM: Internalizing Active Mentalization for Spatial Reasoning

Abstract

When solving complex spatial reasoning tasks, ordinary humans naturally revisit the scenes from task-relevant viewpoints and mentally reconstruct the spatial layouts from those viewpoints. But can Large Vision-Language Models (LVLMs) proactively think in the same way? We term this ability Active Spatial Mentalization (ASM), an intrinsic component of visual intelligence that remains substantially underdeveloped in current LVLMs. To bridge this gap, we introduce SpatialASM, a scalable framework to internalize ASM into LVLMs through training-time spatial simulation. SpatialASM employs an allocentric novel-view generator to ground hypothetical viewpoints in visual evidence and produces two complementary forms of supervision: i) explicit ASM trajectories that teach LVLMs when and how to adaptively invoke view planning and imagination during spatial reasoning, and ii) automatically-constructed perspective-taking tasks that further strengthens viewpoint-conditioned scene imagination. On seven benchmarks spanning relatively perception-oriented spatial understanding to more challenging tasks requiring multi-image, cross-view and viewpoint-transformation reasoning, SpatialASM yields performance gain on LVLMs of four LVLM backbones in spatial reasoning while retaining their original general vision capabilities. The experimental results demonstrate that our approach improves average accuracy across the seven spatial benchmarks. And the synthesis of ASM trajectories provides a scalable pathway to alleviate the scarcity of spatial Chain-of-Thought (CoT) supervision for eliciting active viewpoint exploration and mental scene simulation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Spatial Reasoning, Active Spatial Mentaling, Data Synthesis, Large Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qdNNOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SpatialASM, a framework that leverages generative image models to teach VLMs to select hypothetical viewpoints and imagine corresponding observations, aiming to improve their intrinsic spatial reasoning capabilities. The method consists of two components: view planning and view imagination. It synthesizes reasoning trajectories using a novel-view generator and employs SFT followed by GRPO to internalize these capabilities into VLMs. Experiments across seven spatial reasoning benchmarks and four VLM backbones show modest but consistent improvements in spatial reasoning performance.

Critical strengths
  1. Clear problem formulation and framework design. The paper clearly identifies the limitations of existing VLMs in active viewpoint planning and imagination. The proposed framework is logically organized, with a clear and reasonable methodology.

  2. Useful analysis of existing model limitations. The paper provides empirical observations on the limitations of existing VLMs in spatial mentalization, particularly their difficulty in spontaneously performing viewpoint planning and imagination. This provides some motivation for the proposed approach.

Critical weaknesses
  1. Limited novelty. The central research question has already been extensively explored in prior work on spatial imagination, viewpoint planning, and world-model distillation [1–4]. Although some of these works are cited, the paper does not sufficiently justify its novelty over them. I find the methodological differences largely incremental, without a fundamentally new research insight.

  2. Insufficient evidence for the core claim. The paper claims to internalize Active Spatial Mentalization, yet the performance improvements are relatively modest (+1.2 to +3.5% on average). More importantly, it lacks controlled comparisons demonstrating that these gains arise specifically from learned viewpoint planning and imagination, rather than additional spatial reasoning supervision. This weakens the central claim of the paper.

[1] Zhang et al., World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning, 2026.

[2] Cao et al., SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery, CVPR 2026.

[3] Bigverdi et al., Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models, ECCV 2026.

[4] Li et al., OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping, 2026.

Questions for authors

Beyond incremental differences in implementation or framework design, what fundamentally distinguishes SpatialASM from existing approaches [1–4]? Could the authors provide concrete examples demonstrating what new insight or capability SpatialASM introduces that previous methods have not already explored or established?

[1] Zhang et al., World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning, 2026.

[2] Cao et al., SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery, CVPR 2026.

[3] Bigverdi et al., Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models, ECCV 2026.

[4] Li et al., OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping, 2026.

5 comments
AuthorsOpenReview ↗

Part 1/3: Related Work

Thank you for your careful review and constructive suggestions. We appreciate your recognition of the clear problem formulation, logically organized framework, and useful empirical analysis of existing LVLM limitations. We have revised the manuscript PDF to clarify the training-time role of the novel-view generator and added controlled comparisons using the same spatial QA instances (Table 5). All newly added section titles in the PDF are highlighted in blue. We will further update the PDF within the next two days to incorporate the related work you mentioned. We address your concerns about novelty and the source of the gains below.

W1 / Question: Limited novelty. Beyond incremental differences in implementation or framework design, what fundamentally distinguishes SpatialASM from existing approaches [1–4]? Could the authors provide concrete examples demonstrating what new insight or capability SpatialASM introduces that previous methods have not already explored or established?

Thanks for pointing out this concern. Our contribution lies in internalizing question-conditioned spatial exploration and reasoning through explicit trajectory supervision. While viewpoint planning and imagination have been studied previously, SpatialASM supervises both through the LVLM's own textual reasoning, without an external simulator at inference time. The trajectories start from a spatial question, plan relevant alternative viewpoints, describe the corresponding imagined observations, and use them to derive an answer. We clarify the major differences with the four related works below.

[1] World2VLM. We discuss the distinction from World2VLM in the final paragraph of Appendix A.2, and an example is provided in Figure 4. World2VLM uses a world model to synthesize spatial-QA tasks that teach the relationship between egocentric actions and changes in visual observations. Its supervision provides final answers rather than reasoning trajectories showing how to select useful viewpoints for solving a spatial question.

As shown in Sec. 4.1 and Figure 14, useful viewpoint exploration is not reliably elicited during ordinary reasoning in the evaluated baseline settings. We trained Qwen3-VL-8B on World2VLM's data and observed no accuracy gain over the baseline on MMSI, whereas SpatialASM improved performance by 3.3 points. This comparison is consistent with the value of question-conditioned reasoning supervision, although it does not by itself isolate all differences between the two data pipelines.

Our central contribution is a scalable method for constructing spatial reasoning supervision from existing QA datasets. Rather than only teaching what happens under a given camera motion, our trajectories teach the model to start from a spatial problem, identify alternative viewpoints that could help resolve it, imagine the corresponding observations, and reason toward an answer. Figures 4 and 6 illustrate the difference in supervision format.

[2] SpatialDreamer. SpatialDreamer belongs to world-model-assisted spatial reasoning and resembles our external-ASM setting. We discuss the distinction from such approaches in Sec. 2, Table 2, and Appendix A.3.

SpatialDreamer investigates how an external world model can augment an LVLM with additional visual observations at inference time, whereas SpatialASM uses the world model as a training-time engine to construct supervision for the LVLM's own reasoning. This leads to three practical differences.

First, inference cost. In our reproduction on 160 questions from MindCube, ViewSpatial, and 3DSR, each question required an average of 48 seconds of SVC execution. And tool instructions, generated observations, and multi-round interaction produced an average prompt context of 11.75k tokens. SpatialASM performs textual reasoning without this world-model execution and interaction at inference time.

Second, such a pipeline is not consistently helpful across spatial tasks. The failure case in Figure 8 illustrates a question with a relatively straightforward solution from the input images, where additional generated views are not more informative and can obscure critical evidence. SpatialASM does not mandate an external call and improves over the base model on both perception-oriented and viewpoint-transformation benchmarks.

Finally, SpatialASM is complementary to SpatialDreamer-style tool use. Table 2 shows that combining internalization and external ASM yields higher accuracy than either setting alone on the evaluated subtasks. Improving intrinsic reasoning and supplying external observations can therefore be pursued together.

AuthorsOpenReview ↗

Part 2/3: Related Work

[3] IPT (Imaginative Perception Tokens). The distinction is between imagining a task-prescribed spatial configuration and supervising viewpoint selection conditioned on the spatial question.

IPT constructs three spatial tasks whose imaginative representation is defined by the task instance: (1) Perspective Taking marks the target position and specifies the viewpoint transformation; (2) Path Tracing fixes the path midpoint and queried viewing side; and (3) Multiview Counting uses a top-down bird's-eye-view representation. Our trajectories instead supervise selecting task-relevant viewpoints and using their imagined observations for reasoning.

Most IPT evaluations focus on its three training tasks, with broader transfer evaluated on subsets of 200 MindCube and 170 All-Angles questions. SpatialASM evaluates the resulting LVLMs across seven spatial benchmarks while also checking general visual capabilities. This provides evidence for the usefulness of our supervision across different spatial tasks, without requiring the LVLM to generate intermediate images.

[4] OmniView-Space. OmniView-Space is a closely related concurrent work. Both approaches internalize question-conditioned spatial reasoning, but differ in the intermediate reasoning process they supervise. OmniView-Space teaches VLMs to select a query-aligned reference frame and construct a BEV cognitive map or textual spatial graph. SpatialASM instead teaches the model to select task-relevant allocentric viewpoints, imagine what they would see, and use these imagined observations to answer the question. Our distinction is therefore in viewpoint-based intermediate supervision, rather than question-conditioned reasoning itself.

AuthorsOpenReview ↗

Part 3/3: Controlled Experiments

W2: Insufficient evidence for the core claim. The paper claims to internalize Active Spatial Mentalization, yet the performance improvements are relatively modest (+1.2 to +3.5% on average). More importantly, it lacks controlled comparisons demonstrating that these gains arise specifically from learned viewpoint planning and imagination, rather than additional spatial reasoning supervision.

We appreciate this useful comment. We added controlled experiments on Qwen3-VL-4B by training it with the same QA instances in SpatialASM-66k but without ASM trajectories under different training strategies: answer-only SFT, standard-CoT SFT, and conventional-QA GRPO. Standard-CoT SFT uses the same teacher model used to synthesize ASM trajectories to generate CoT for SFT training. We additionally report our ASM-trajectory SFT model (the "SFT with filter" setting in the original Table 4) and the complete SpatialASM-4B model. The additional SFT and GRPO training follows the configurations described in Sec. 4.

MethodsMindCubeMMSISpa457-6dEmbSpaAVG
Qwen3-VL-4B33.928.037.272.342.9
+ Answer-only SFT39.028.126.869.540.9
+ Standard-CoT SFT31.528.733.176.042.3
+ Conventional-QA GRPO33.926.939.676.244.2
+ ASM-trajectory SFT38.729.442.674.646.3
SpatialASM-4B39.129.242.876.546.9

Simply adding the same 66k QA instances, whether through answer-only/standard-CoT SFT or conventional-QA GRPO, does not reproduce the average gains obtained with ASM-trajectory supervision. Some baselines regress considerably on individual benchmarks, whereas ASM-trajectory SFT improves over the base model on all four and achieves the strongest overall single-stage performance. Since these comparisons use the same spatial QA instances, the improvement cannot be explained by the additional QA instances alone. These results support the benefit of our proposed ASM supervision for spatial reasoning.

Benchmark details. The three leftmost benchmarks require integrating incomplete spatial evidence and reasoning through viewpoint exploration or transformation, while EmbSpa targets more general spatial understanding that can often be resolved from the observed input views.

Answer-only SFT. This strategy shows uneven transfer: it improves MindCube but degrades on Spatial457-6d and EmbSpa. Thus, it does not reliably improve performance across the evaluated spatial tasks.

Standard-CoT SFT and conventional-QA GRPO. Both rely on LVLM-generated reasoning, produced either by the teacher or by the model's own rollouts. Their limited overall gains are consistent with the importance of grounding intermediate reasoning in spatial observations. We treat this as an interpretation of the results rather than a direct measurement of reasoning-trace quality.

We also provide targeted viewpoint evaluations in Table 6. When changing only the planner and keeping the NVG and final Qwen3-VL-8B reasoner fixed, SpatialASM-8B improves downstream accuracy from 42.0 to 45.5 on Among and from 45.5 to 48.5 on Around. On tasks with a specified target perspective, scores increase from 43.3 to 44.3 on ViewSpatialBench-PersRel and from 25.9 to 30.4 on 3DSRBench-MultiViewTo. These results support improved planning utility and viewpoint-conditioned reasoning; adaptive invocation and internal imagination fidelity are not separately verified by these evaluations.

Reviewer_NyE9OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SpatialASM, a data generation and fine-tuning framework for VLMs to enhance the internal "active spatial mentalization" capability required for viewpoint-aware spatial understanding. SpatialASM includes an allocentric novel-view generator assisted by the Geometry Perturbation (GeoP) metric to measure the amount of perturbation introduced by the view synthesis. It also involves trajectory synthetic pipeline where the generated reasoning trajectory is used to supervise the model in a richer manner compared to simple QA format. Finally, a view-imagination RL stage enables the VLM to internally process the input views through ASM-aware reasoning mechanism. The advantage of SpatialASM is shown through performance improvement across multiple spatial benchmarks, outperforming the base Qwen models after SFT and RL post-training.

Critical strengths
  • Consistent improvements across model sizes. SpatialASM improves performance across Qwen backbones of multiple sizes, suggesting that its benefits generalize across model scales.

  • Reasoning-aware training format for spatial reasoning. The authors suggest reasoning trajectories and multi-step spatial reasoning beyond simple question–answer pairs. This format provides a more natural basis for training VLMs to internalize ASM capabilities.

Critical weaknesses
  • The writing is not clear and coherent. The paper seems more as a collection of engineering techniques for improving spatial reasoning benchmark scores rather than a unified approach grounded in specific observations about how and when VLMs fail. The paper should emphasize its core observations more clearly and elaborate on how each component is designed to address the fundamental failures of such models.

  • The gains in Table 1 are not convincing given the method’s complexity. SpatialASM improves the average scores by only +1.2 over the base model Qwen3.5-9B, while using a sophisticated framework from novel-view synthetic to RL post-training. These do not justify the value of bringing in additional complexity. I believe a simple SFT and RL post-training using some additional spatial reasoning data would lead to comparable or better results. Could the authors provide more discussions on why such a sophisticated framework is necessary?

  • The role of Table 2 is not clear. SpatialASM aims to internalize ASM capabilities within VLMs, yet Table 2 emphasizes that providing an external ASM further improves performance. This result does not demonstrate stronger internal spatial understanding of the model. It is obvious that providing addditional views would improve performance as it is supplying more visual information. The paper should distinguish the benefits of internalized spatial reasoning from those of external input and include controls that isolate these effects.

  • The meaning of “verifiable” in the RL data generation is unclear. Does verification apply only to the final answer, or also to the intermediate reasoning? If only the final answer is being verified, a correct response does not guarantee that the reasoning is also "spatially correct". The authors should clarify what is verified, how verification is performed, and whether the training procedure can reward correct answers reached through incorrect spatial reasoning.

  • The evaluation omits important benchmarks. The paper does not evaluate on VSI-Bench, which is a standard benchmark in this area. ReVSI [1] is also another option, since it is an improved version of VSI-Bench.

  • Several closely related works and important baselines are missing. Authors should discuss these works and also include comparions with some:

    • 3DThinker [2] reports strong accuracy on MindCube and also enables VLMs to "imagine" additional viewpoints.

    • Mirage [3] and Ross3D [4] try to internalize spatial imagination capability into VLMs through generative or reconstructive training objectives.

    • SpaceTools [5] is a representative work on using vision tools for spatial reasoning.


[1] ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning, Zhang et al., ICML 2026

[2] Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views, Chen et al., CVPR 2026

[3] Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens, Yang et al., CVPR 2026

[4] Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness, Wang et al., ICCV 2025

[5] SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL, Chen et al., CVPR 2026

Questions for authors
  • Why were the original MindCube checkpoints (SFT and RL) not considered as a baseline?

  • As Table 2 shows, providing additional view information through external ASM already provides notable performance improvement to the base Qwen model. Given this, why is it practically necessary for the VLMs to further have "internal" ASM ability?

7 comments
AuthorsOpenReview ↗

Part 1/4: Baseline Failure Modes Analysis

Thank you for your detailed review and helpful suggestions. We appreciate your recognition of the consistent improvements across model sizes and the value of reasoning-aware training beyond simple question–answer pairs. We have revised the manuscript PDF to clarify the training-time simulation framework and added the controlled training comparisons in Table 5. All newly added section titles in the PDF are highlighted in blue. And our PDF has incorporated the related work you mentioned.

Below, we explain the failure modes motivating the method, clarify the role of internalization and answer verification, and report the additional benchmark and baseline results.

W1: The writing is not clear and coherent. The paper should emphasize its core observations more clearly and elaborate on how each component addresses the fundamental failures of VLMs.

We appreciate the advice that the paper should articulate the core observations of VLM failure modes more clearly. Two major problems motivate our design: insufficient proactive viewpoint exploration and unreliable viewpoint-conditioned imagination.

Failure modes. Our preliminary analysis (Sec. 4.1) suggests that explicitly prompting LVLMs to invoke a world model can improve accuracy, but this does not establish that they spontaneously initiate useful exploration during ordinary reasoning. Figure 14 in Appendix E.4 illustrates the first problem: the baseline reasons under fixed input views, whereas SpatialASM imagines a relevant perspective to resolve the spatial relationship. Figure 12 illustrates the second problem: when the target perspective is specified, the baseline can misinterpret the scene from that viewpoint. These cases illustrate the motivation, rather than quantify the prevalence of each failure mode.

Our components. ASM trajectory supervision teaches viewpoint planning and the use of imagined observations in reasoning. View-imagination RL further strengthens the association between a specified viewpoint and its corresponding observation. Our objective is to develop spatial reasoning capabilities, evaluated through performance on diverse benchmarks. The source spatial questions for our SFT reasoning supervision come from HoliSpatial, rather than being separately curated for each evaluation benchmark.

AuthorsOpenReview ↗

Part 2/4: Complexity Clarification and Controlled Comparison

W2: The gains in Table 1 are not convincing given the method's complexity. Could simple SFT and RL post-training with additional spatial data achieve comparable results?

Clarification on complexity. We agree that our pipeline involves multiple components. However, it is useful to distinguish the offline data-construction cost from the complexity of adapting and deploying a model. A user can directly use our constructed training data for LVLM post-training. At inference time, the model provides textual CoT without invoking external tools or world models. The offline cost is therefore reusable across backbones, while no novel-view generator is required at deployment.

In contrast, 3DThinker and Mirage introduce specialized latent tokens and representation-alignment training, whereas SpatialASM does not modify the VLM architecture or its latent representation space. Our textual reasoning also avoids the world-model execution and multi-round interaction required by inference-time tool pipelines. We do not claim that offline data construction is cost-free.

Comparison with simple post-training. We added controlled experiments on Qwen3-VL-4B by training it with the same QA instances in SpatialASM-66k but without ASM trajectories under different training strategies: answer-only SFT, standard-CoT SFT, and conventional-QA GRPO. Standard-CoT SFT uses the same teacher model used to synthesize ASM trajectories to generate CoT for SFT training. We additionally report our ASM-trajectory SFT model (the "SFT with filter" setting in the original Table 4) and the complete SpatialASM-4B model. The additional SFT and GRPO training follows the configurations described in Sec. 4.

MethodsMindCubeMMSISpa457-6dEmbSpaAVG
Qwen3-VL-4B33.928.037.272.342.9
+ Answer-only SFT39.028.126.869.540.9
+ Standard-CoT SFT31.528.733.176.042.3
+ Conventional-QA GRPO33.926.939.676.244.2
+ ASM-trajectory SFT38.729.442.674.646.3
SpatialASM-4B39.129.242.876.546.9

Simply adding the same 66k QA instances, whether through answer-only/standard-CoT SFT or conventional-QA GRPO, does not reproduce the average gains obtained with ASM-trajectory supervision. Some baselines regress considerably on individual benchmarks, whereas ASM-trajectory SFT improves over the base model on all four and achieves the strongest overall single-stage performance. Since these comparisons use the same spatial QA instances, the improvement cannot be explained by the additional QA instances alone. These results support the benefit of our proposed ASM supervision for spatial reasoning.

The three leftmost benchmarks require viewpoint exploration or transformation and integration of incomplete spatial evidence; EmbSpa emphasizes more general spatial understanding. Answer-only SFT shows uneven transfer across these benchmarks. Standard-CoT SFT and conventional-QA GRPO also fail to reproduce the overall gains of ASM-trajectory supervision. We regard the differences in supervision as a plausible explanation, rather than attributing all baseline failures to a single cause.

AuthorsOpenReview ↗

Part 3/4: Explanation of Paper Details and Evaluation on VSI-Bench

W3 / Q2: Table 2 does not clearly distinguish internalized reasoning from the benefit of additional views. Why is internal ASM practically valuable when external ASM already improves performance?

We apologize for the lack of clarity. Table 2 is intended to demonstrate the complementary relationship between internalization and external tool use. Our goal is not to replace tool use, but to improve the model's own spatial reasoning capability while remaining compatible with external evidence.

The settings in Table 2 separate the practical benefits as follows:

BenefitCorresponding comparison in Table 2
External ASMQwen3-VL-8B Vanilla vs. Qwen3-VL-8B + External ASM
Post-training without external viewsQwen3-VL-8B Vanilla vs. SpatialASM-8B Vanilla
Combined settingQwen3-VL-8B Vanilla vs. SpatialASM-8B + External ASM

The vanilla comparison evaluates both models using the original input images, so the gain in that setting cannot be attributed to additional rendered views at inference time. The combined setting shows that external tool use can still bring gains after post-training. These are comparisons of practical performance, rather than standalone proof of the internal mechanism.

Moreover, providing generated observations does not remove the need to decide which viewpoints are useful or how their evidence should be interpreted. Those decisions still require LVLM reasoning. Our additional planner-controlled evaluation keeps the NVG and final reasoner fixed while replacing only the planner; downstream accuracy improves from 42.0 to 45.5 on Among and from 45.5 to 48.5 on Around. This more specifically supports the usefulness of the trained model's viewpoint plans.

W4: What does "verifiable" mean in RL data generation? Can correct answers reached through incorrect spatial reasoning be rewarded?

In our paper, "verifiable" means that the answer index is determined automatically during RL task construction, without a separate annotation of that index by humans or an external model. The model's final answer can then be checked against the constructed label. This does not mean that the generated observation or the intermediate reasoning is independently verified.

Verification applies only to the final answer, not to the intermediate reasoning process. Therefore, an incorrect reasoning trace that reaches the correct option can still receive a positive answer reward. Our primary contribution to explicit reasoning supervision lies in the SFT stage, where we synthesize ASM CoT to supervise the plan–imagine–reason process. Reliable process-level rewards for spatial reasoning remain a limitation and a future direction.

W5: The evaluation omits VSI-Bench; ReVSI is another option.

We thank the reviewer for pointing out the missing evaluation on video spatial reasoning. Our original workflow primarily focused on single-image and multi-image spatial tasks. We now add an evaluation on VSI-Bench with the following protocol: we sample 16 frames from each video and prompt the model to reason about the question and provide its final answer within <answer></answer> tags.

ModelScore
Qwen3-VL-4B45.06
SpatialASM-4B46.29 (+1.23)
Qwen3-VL-30BA3B46.89
SpatialASM-30BA3B47.41 (+0.52)

The results provide additional evidence of transfer to video-based spatial reasoning under this 16-frame evaluation setting. We report VSI-Bench here; ReVSI is not included in this evaluation.

Reviewer_MBReOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SpatialASM, a framework for internalizing Active Spatial Mentalization (ASM) into LVLMs through viewpoint planning and imagination. The method uses an allocentric novel-view generator to synthesize spatial reasoning trajectories for supervised fine-tuning, followed by GRPO on automatically constructed perspective-taking tasks. A geometry-based metric (GeoP) is introduced to filter generated observations. Experiments across four LVLM backbones and seven spatial reasoning benchmarks demonstrate consistent accuracy improvements while largely preserving general visual capabilities, without requiring external world models during inference.

Critical strengths

Meaningful and practical framework. The paper introduces an intuitive formulation of Active Spatial Mentalization and proposes a scalable pipeline for generating spatial reasoning supervision. Internalizing spatial imagination without requiring external world models at inference time is practically valuable.

Consistent empirical improvements. The method demonstrates improvements across four LVLM backbones and seven spatial reasoning benchmarks, while largely maintaining general visual capabilities. The ablation studies also provide evidence for the complementary benefits of SFT and RL.

Critical weaknesses

W1 — Insufficient validation of ASM internalization. The experiments mainly demonstrate improvements in spatial QA accuracy without directly measuring viewpoint-planning quality, imagination fidelity, or adaptive invocation. It remains unclear whether the gains genuinely arise from internalizing ASM rather than general spatial reasoning improvements.

W2 — Missing essential training baselines. The paper lacks comparisons with simpler alternatives, such as answer-only spatial SFT, standard spatial CoT SFT, and conventional spatial QA GRPO under comparable data and compute budgets. Without these baselines, it is difficult to isolate the specific contribution of the proposed ASM framework.

Questions for authors

Q1. Can the authors provide quantitative evidence that the trained models actually acquire ASM capabilities, such as viewpoint-planning accuracy, imagination fidelity, or appropriate invocation of spatial imagination?

Q2. How does SpatialASM compare with standard spatial CoT SFT or answer-only SFT combined with conventional GRPO under comparable training budgets?

Q3. Has GeoP been validated against ground-truth geometric consistency or viewpoint fidelity? How does the filtering strategy handle geometrically correct views that reveal previously unobserved regions?

Q4. How do the authors ensure that the automatically constructed RL questions have unique and correct answers, given that multiple generated viewpoints may produce similar observations?

5 comments
AuthorsOpenReview ↗

Part 1/3: Decoupled Capabilities

Thank you for your careful review and constructive questions. We appreciate your recognition of the meaningful and practical framework, the value of internalizing spatial imagination without an inference-time world model, and the consistent empirical improvements across model scales and benchmarks. We have revised the manuscript PDF to clarify the training-time simulation framework and include the controlled training comparisons (Table 5) and targeted capability results (Table 6). Below, we address your questions about ASM capabilities, training baselines, GeoP, and RL task construction.

W1 / Q1: Can the authors provide quantitative evidence that the trained models actually acquire ASM capabilities, such as viewpoint-planning accuracy, imagination fidelity, or appropriate invocation of spatial imagination?

We thank the reviewer for this insightful suggestion. We agree that overall accuracy alone does not fully reveal which component of ASM is improved. Directly evaluating the appropriate invocation of ASM is challenging because it is difficult to define a ground-truth label indicating whether a particular question must invoke ASM. Following the reviewer's suggestion, we conducted two targeted evaluations of viewpoint-planning quality and viewpoint-conditioned reasoning.

Viewpoint-planning ability. We follow the external-ASM protocol in Sec. 4.1 while changing only the model used in the viewpoint-planning stage. Specifically, we use either Qwen3-VL-8B or SpatialASM-8B to plan task-relevant viewpoints. The planned viewpoints are rendered using the same SpatialASM-NVG, and the resulting observations are passed to the same Qwen3-VL-8B for final reasoning. The evaluation uses the same MindCube subtasks as in Sec. 4.1 (200 questions from Among and 200 from Around). SpatialASM-8B achieves higher downstream accuracy on both subtasks as the viewpoint planner.

Evaluation taskQwen3-VL-8B plannerSpatialASM-8B planner
MindCube–Among42.045.5
MindCube–Around45.548.5

Viewpoint-imagination ability. To exclude the influence of viewpoint selection, we evaluate the models on two subtasks in which the target perspective is explicitly specified in the question: Person Perspective-Relative Direction from ViewSpatial-Bench and multi-object-viewpoint-towards-object from 3DSRBench. These tasks do not require the models to plan which viewpoint to adopt. Instead, they require them to infer the relative spatial configuration from the designated perspective. SpatialASM-8B outperforms Qwen3-VL-8B on both subtasks, suggesting stronger viewpoint-conditioned spatial reasoning.

Evaluation taskQwen3-VL-8BSpatialASM-8B
ViewSpatialBench-PersRel43.344.3
3DSRBench-MultiViewTo25.930.4

These evaluations provide behavioral evidence for the intended ASM capabilities; they do not directly measure the fidelity of internal imagined representations or separately establish appropriate invocation.

AuthorsOpenReview ↗

Part 2/3: Controlled Experiments

W2 / Q2: How does SpatialASM compare with standard spatial CoT SFT or answer-only SFT combined with conventional GRPO under comparable training budgets?

We added controlled experiments on Qwen3-VL-4B by training it with the same QA instances in SpatialASM-66k but without ASM trajectories under different training strategies: answer-only SFT, standard-CoT SFT, and conventional-QA GRPO. Standard-CoT SFT uses the same teacher model used to synthesize ASM trajectories to generate CoT for SFT training. We additionally report our ASM-trajectory SFT model (the "SFT with filter" setting in the original Table 4) and the complete SpatialASM-4B model. The additional SFT and GRPO training follows the configurations described in Sec. 4.

MethodsMindCubeMMSISpa457-6dEmbSpaAVG
Qwen3-VL-4B33.928.037.272.342.9
+ Answer-only SFT39.028.126.869.540.9
+ Standard-CoT SFT31.528.733.176.042.3
+ Conventional-QA GRPO33.926.939.676.244.2
+ ASM-trajectory SFT38.729.442.674.646.3
SpatialASM-4B39.129.242.876.546.9

Simply adding the same 66k QA instances, whether through answer-only/standard-CoT SFT or conventional-QA GRPO, does not reproduce the average gains obtained with ASM-trajectory supervision. Some baselines regress considerably on individual benchmarks, whereas ASM-trajectory SFT improves over the base model on all four and achieves the strongest overall single-stage performance. Since these comparisons use the same spatial QA instances, the improvement cannot be explained by the additional QA instances alone. These results support the benefit of our proposed ASM supervision for spatial reasoning.

The table reports conventional SFT and GRPO individually, rather than a conventional SFT-plus-GRPO sequence. It therefore isolates the benefit of ASM-trajectory SFT among the evaluated single-stage alternatives, without claiming to cover all combinations of conventional post-training.

AuthorsOpenReview ↗

Part 3/3: GeoP Validation and RL-candidate Explanation

Q3: Has GeoP been validated against ground-truth geometric consistency or viewpoint fidelity? How does the filtering strategy handle geometrically correct views that reveal previously unobserved regions?

Thank you for this detailed question. We evaluated GeoP on the first 35k ground-truth novel-view samples of SpatialASM-edit140k (approximately one quarter). The resulting median GeoP is 0.114, which is comparable to the range reported for generated views in Table 3. This provides a sanity check on the scale of GeoP for genuine novel views, rather than a calibration of its ability to distinguish correct and incorrect geometry.

We acknowledge the limitation of GeoP. A geometrically correct view can receive an elevated GeoP when it reveals regions not observed in the input images. Consequently, correct views containing substantial unobserved content may be rejected, increasing false negatives. For this reason, GeoP is used as a rejection criterion, rather than a training loss or a fine-grained quality-ranking metric. It is a model-dependent proxy, and a low score does not by itself guarantee target-viewpoint fidelity. Table 4 shows the practical benefit of GeoP-based filtering for both SFT and RL. For SFT trajectories, the answer-consistency check provides an additional filter.

Q4: How do the authors ensure that the automatically constructed RL questions have unique and correct answers, given that multiple generated viewpoints may produce similar observations?

We apologize that the candidate-viewpoint sampling procedure was not described in sufficient detail. In practice, to reduce the likelihood that multiple candidates produce similar observations, we enforce spatial and directional diversity among the sampled allocentric viewpoints.

Specifically, for an anchor image of width W and height H, we impose three constraints: (1) the left, middle, and right thirds of the anchor image must each contain at least one candidate location; (2) the left, backward, and right viewing directions must each appear at least once; and (3) the Euclidean distance between any two candidate locations must be at least 0.2 min(W,H).

The answer index is determined during construction, and GeoP filters the selected target view for geometric consistency. The sampling constraints reduce ambiguity but do not guarantee visually distinct observations in symmetric or repetitive scenes. Thus, automatically obtaining an answer index is distinct from guaranteeing perceptual unambiguity, and residual ambiguity remains a limitation of the constructed tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.