acceptodds
Withdrawn from ICLR 2027

URUQI: Learning Spatial Cognition from Visual Experience

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatial intelligence requires maintaining a coherent understanding of the world as the embodied agent moves. Like humans, the agent must use its own motion to interpret changes across observations and update object locations and spatial relations accordingly. Despite spatial post-training having substantially broadened the spatial intelligence of vision-language models (VLMs), they still struggle with two atomic spatial capabilities: tracking self-motion and mapping the surrounding world during motion. To address this gap, we provide dense multi-turn supervision over interleaved atomic capabilities within each training episode, mimicking the visual experience of a continuously moving agent that reasons as it observes. To scale this up, we synthesize 11,738 motif-driven camera trajectories over a broad range of 3D scenes, supporting self-motion tracking, persistent object mapping, and rich spatial operations within each visual experience. By training models to reason over these atomic questions, our URUQI-Syn-8B improves accuracy from 15.84% to 47.73% on our URUQI benchmark comprising 52k questions across 2.7k episodes. URUQI-SI-Mix-8B further reaches 50.41%, comparable to the 50.08% achieved by GPT-6 Astra. Trained solely on our synthesized data, URUQI-Syn-8B achieves an average relative accuracy improvement of 17.13% over its InternVL3-8B backbone across three external spatial benchmarks. These results highlight continuous visual experience as a scalable source of supervision for developing spatial cognition in VLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Embodied AI, Vision-Language Models, Spatial intelligence

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_x9K6OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes URUQI, a framework for learning spatial cognition in VLMs from visual experience. It introduces a data-generation pipeline that produces motif-driven camera trajectories in simulated 3D scenes and derives supervision for self-motion tracking, persistent object mapping, and operations over spatial state. Trained on URUQI-600k, the InternVL3-8B–based URUQ-8B achieves state-of-the-art performance on the proposed URUQI benchmark and improves over its backbone on VSI-Bench, MMSI-Bench, and MindCube-tiny.

Critical strengths
  1. Scalable data-generation pipeline. This work introduces a scalable pipeline that generates motif-driven camera trajectories in simulated 3D scenes and uses privileged camera poses, object identities, and visibility annotations to derive supervision for self-motion tracking, persistent object mapping, and operations over spatial state.
Critical weaknesses
  1. Insufficient evidence supporting the main contribution. The paper presents the joint organization of self-motion tracking, persistent object mapping, and spatial-state operations within visual experience as its main contribution. In practice, however, this idea is implemented primarily through data construction, while model training relies on a standard answer-token prediction objective. the paper does not sufficiently establish what the proposed organization of supervision contributes beyond existing spatial QA data-generation approaches. Furthermore, the lack of direct comparisons with existing data pipelines (such as Holi-Spatial (ICML 2026)) makes it difficult to assess the distinctiveness of the proposed method.

  2. Modest transfer gains and incomplete comparisons. The improvements on external spatial reasoning benchmarks are relatively modest. Compared with SenseNova-SI-1.5, URUQI-SI-Mix-8B achieves overall score gains of 0.55, 2.70, and 0.57 on VSI-Bench, MMSI-Bench, and MindCube-tiny, respectively. Moreover, its training data include 100k examples randomly sampled from SenseNova-SI-8M, making it difficult to isolate the contribution of URUQI supervision. Comparisons with recent spatial reasoning approaches, including 3DThinker (CVPR 2026), GeoThinker (ICML 2026), and Holi-Spatial (ICML 2026), would further clarify the method’s relative effectiveness.

Questions for authors

Please see the weaknesses above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer x9K6

Thank you for your valuable feedback and for recognizing the scalability of our data-generation pipeline. We clarify our key contributions and provide additional experimental evidence and comparisons below.

W1. Contribution of Interleaved Spatial Supervision and Benchmark

"The paper does not sufficiently establish what the proposed organization of supervision contributes beyond existing spatial QA data-generation approaches." "Model training relies on a standard answer-token prediction objective." "The lack of direct comparisons with existing data pipelines (such as Holi-Spatial) makes it difficult to assess the distinctiveness of the proposed method."

Thank you for raising these questions. We would like to clarify that URUQI contributes both an evaluation of coherent spatial understanding and an interleaved supervision approach designed to develop this capability.

(1) URUQI benchmark evaluates coherent spatial understanding beyond conventional spatial QA performance. Existing spatial benchmarks primarily evaluate whether models answer spatial questions correctly. In contrast, URUQI uses temporally connected queries within continuous visual experiences to assess whether models can track their own motion, maintain previously observed object locations, and update spatial relationships as the observer moves. This distinction reveals a substantial capability gap that previous spatial benchmarks fail to capture. SenseNova-SI-1.5 achieves 92.00% on MindCube-tiny but only 17.02% on URUQI, whereas GPT-6 Astra reaches 50.08% without URUQI-specific training (Table 1). These results show that strong performance on existing spatial QA benchmarks does not necessarily imply coherent spatial understanding across observations.

(2) Our training contribution lies in how spatial supervision is organized within continuous visual experiences. Existing pipelines, such as Holi-Spatial, provide rich geometric and relational supervision through spatial QA. URUQI interleaves temporally grounded QA within a shared context. This organization allows successive questions to build on the same observation history, linking observer motion to changes in object locations and spatial relations. Our same-QA single-turn comparison isolates the benefit of this episodic organization, with interleaved training outperforming independent single-turn training across all four benchmarks (Table R1).

Training ConfigurationURUQIVSI-BenchMMSI-BenchMindCube-tiny
M1+M2+M3 (Single-turn)44.2846.9631.2042.29
URUQI-Syn-8B (Interleaved Episode)47.7347.2633.1045.90

To further address the suggested comparison with Holi-Spatial, we also evaluate its officially released 2M-QA checkpoint on URUQI and external benchmarks (please see Table R4 below).

We have incorporated these ablations and the single-turn comparison into Section 4.4 and Table 3 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer x9K6 (Part 2)

W2. External Transfer and Comparisons with Recent Spatial approaches

"The improvements on external spatial reasoning benchmarks are relatively modest." "Its training data include 100k examples randomly sampled from SenseNova-SI-8M, making it difficult to isolate the contribution of URUQI supervision." "Comparisons with recent spatial reasoning approaches, including 3DThinker, GeoThinker, and Holi-Spatial, would further clarify the method's relative effectiveness."

Thank you for suggesting these comparisons. Our newly added SenseNova-100k-only ablation demonstrates substantial additional gains from URUQI supervision, and we further compare our results with the suggested recent methods.

URUQI provides substantial transferable gains beyond conventional spatial QA supervision. With the same InternVL3-8B initialization and SenseNova-100k subset, adding URUQI improves all three external benchmarks (+4.13/+9.80/+9.05) and URUQI accuracy (+30.49) (as shown in Table R2 below).

Training DataURUQIVSI-BenchMMSI-BenchMindCube-tiny
SenseNova-100k only (New)17.8653.4330.0046.95
URUQI + SenseNova-100k48.3557.5639.8056.00
Gain+30.49+4.13+9.80+9.05

These gains show that visual-experience supervision provides additional spatial learning beyond exposure to task-aligned QA data alone. Moreover, URUQI-Syn-8B independently improves all three external benchmarks (+5.12/+5.30/+7.71) using only simulated visual experiences, without their training data (Table 2).

We have added the SenseNova-100k-only ablation to Table 2(b) and discussed its comparison with URUQI-Mix-8B in Section 4.3.

(2) Our transfer gains are competitive with recent spatial reasoning methods. We compare the reported results of Holi-Spatial, GeoThinker, and 3DThinker on MMSI-Bench, which has no official training set (as shown in Table R3 below).

MethodBackboneMain ApproachMMSI Base → TrainedGain
Holi-SpatialQwen3-VL-8BVideo-derived 3D spatial QA31.10 → 32.60+1.50
GeoThinkerQwen3-VL-8BActive geometric feature integration28.80 → 30.90+2.10
GeoThinkerQwen2.5-VL-7BActive geometric feature integration26.80 → 31.70+4.90
3DThinkerQwen2.5-VL-7B3D latent alignment and outcome optimization26.90 → 43.30+16.40
URUQI-Syn-8BInternVL3-8BInterleaved visual-experience supervision27.80 → 33.10+5.30
URUQI-Mix-8BInternVL3-8BVisual experiences + spatial QA27.80 → 39.80+12.00

We further evaluate the released HoliSpatial-2M-QA-Qwen3-VL-8B to directly examine the spatial capabilities learned from different supervision approaches. Table R4 below compares Holi-Spatial with our newly trained URUQI-only model based on the Qwen3-VL-8B backbone.

ModelURUQIVSI-BenchMMSI-BenchMindCube-tiny
Holi-Spatial (Qwen3-VL-8B)17.5846.8432.6054.38
URUQI-Only (Qwen3-VL-8B)54.7654.5634.7044.38

The comparison reflects different training coverage. Holi-Spatial benefits from extensive spatial QA supervision, while our Qwen3-VL-8B uses only synthetic visual experiences and achieves substantially better coherent spatial understanding (Table R4). Due to limited time, we have not trained a Qwen3-VL-8B Mix variant. Notably, our InternVL3-8B results demonstrate that adding task-aligned spatial QA data can further improve MindCube performance.

(3) More importantly, improvements on spatial QA benchmarks do not necessarily imply coherent spatial understanding. SenseNova-SI-1.5 achieves 92.00% on MindCube-tiny but only 17.02% on URUQI, whereas GPT-6 Astra reaches 50.08% without URUQI-specific training (Table 1). This contrast reveals a capability gap that conventional spatial QA benchmarks do not adequately capture.

Our training addresses this gap while further improving existing spatial QA performance. URUQI raises SenseNova-SI-1.5's accuracy from 17.02% to 50.41% (+33.39), alongside improvements on all three external benchmarks. Thus, URUQI not only reveals this previously underexplored gap, but also enables models to develop coherent spatial understanding alongside their existing spatial QA capabilities.

Dear Reviewer x9K6,

As the discussion period is coming to an end, we would greatly appreciate your feedback on whether our clarifications have addressed your main concerns.

Thank you again for your time and consideration.

Best regards,

Submission 7770 Authors

Reviewer_XnT7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents URUQI, a vision-language model (VLM) post-training method that uses continuous visual experience, to jointly supervise self-motion, persistent object mapping, and downstream spatial reasoning within continuous visual experiences, for improving spatial cognition. The core contribution is a data generation engine that uses reusable motifs in 3D scenes to generate visual experience. It searches for feasible camera trajectories and renders RGB observations along with depth and instance identities. Finally, each trajectory is verified before being compiled into supervision over the three stages mentioned above. After training on those synthetically generated data, the model improves the accuracy from roughly 15% to 47% on their benchmark. It also improves by 17% on external spatial benchmarks when compared to its base model, InternVL3-8B. Overall, this paper demonstrates a method to improve VLMs' spatial cognition by training on synthetic data.

Critical strengths
  1. A simple method to improve performance a lot, with evaluations on frontier proprietary models. On their own dataset, it improves the accuracy from 15% to 47%-50%, while the frontier proprietary model GPT-6 achieves around 50%, which is on par. I think integrating frontier model results makes the paper much stronger compared to only less capable models. The evaluation suite also includes external benchmarks, including VSI-Bench, MMSI-Bench, and MindCube-tiny. Strong results on these 3 external benchmarks show the post-training process's generalizability.
Critical weaknesses
  1. Lack of an ablation study to show the contribution of each supervision to the performance gain. The paper lacks an ablation study, which is its main weakness. Since the paper emphasizes the three supervisions/stages - self-motion (M1), persistent object mapping (M2), and operations over spatial state (M3), it is crucial to have an ablation study to show the contribution of each stage. From the current results, one could interpret that only self-motion or only persistent object mapping is important to this post-training process. I will raise the score if you show the ablation study and the results support your central claims.
Questions for authors
  1. Are you using LoRA fine-tuning or full-weight training? I think it is full-weight training since you're using 16 NVIDIA H800 GPUs and a batch size of 192, but I want to double-check.

  2. What is the reasoning effort used in GPT-6 Astra evaluation?

  3. Will this post-training process cause catastrophic forgetting of the original model's capabilities beyond spatial cognition? If you didn't run this experiment, that's fine, but if you do notice anything, I think this insight is interesting and important to the post-training and continual learning community as well.

  4. Minor suggestion: I think Figure 2 should be the first figure of the paper. The current Figure 1 is actually making the paper harder to understand.

4 comments
AuthorsOpenReview ↗

Response to Reviewer XnT7 (Part 1)

W1. Contribution of Spatial Supervision

Reviewer: "The paper lacks an ablation study to show the contribution of each supervision to the performance gain. [...] I will raise the score if you show the ablation study and the results support your central claims."

Thank you for your suggestion. We have added M3-only, w/o M1, and w/o M2 ablations to examine the contribution of spatial supervision, along with a same-QA single-turn comparison to evaluate episodic organization (as shown in Table R1 below).

Training ConfigurationURUQIVSI-BenchMMSI-BenchMindCube-tiny
M3-only29.0946.5831.1044.38
M2+M3 (w/o M1)35.9846.3030.2041.52
M1+M3 (w/o M2)45.2345.5829.9045.43
M1+M2+M3 (Single-turn)44.2846.9631.2042.29
URUQI-Syn-8B (Interleaved Episode)47.7347.2633.1045.90

(1) Joint supervision achieves the best performance across all four benchmarks. Removing M1 reduces URUQI accuracy from 47.73% to 35.98%, removing M2 reduces it to 45.23%, while M3-only achieves 29.09%. The contributions also extend to external benchmarks: removing M1 reduces MindCube by 4.38 points, while removing M2 reduces MMSI by 3.20 points.

(2) Interleaved episodic training provides additional benefits beyond isolated single-turn QA. With the same M1, M2, and M3 questions, interleaved training outperforms single-turn training across all four benchmarks. These results support our central insight that interleaving self-motion tracking, persistent object mapping, and spatial operations within the same visual experience helps models learn their underlying spatial dependencies and maintain coherent spatial understanding.

We have incorporated these ablations and the single-turn comparison into Section 4.4 and Table 3 of the revised manuscript.

(3) The learned spatial capabilities also provide substantial gains on external benchmarks. To further examine their transferability, we compare URUQI-Mix-8B against our newly added SenseNova-100k-only ablation (as shown in Table R2 below).

Training DataURUQIVSI-BenchMMSI-BenchMindCube-tiny
SenseNova-100k only17.8653.4330.0046.95
URUQI data + SenseNova-100k48.35(+30.49)57.56 (+4.13)39.80 (+9.80)56.00 (+9.05)

We have added the SenseNova-100k-only ablation to Table 2(b) and discussed its comparison with URUQI-Mix-8B in Section 4.3.

Q1 & Q2. Training and Evaluation Settings

Q1: "Are you using LoRA fine-tuning or full-weight training?"

Regarding the training configuration, We use full-parameter fine-tuning. We use a learning rate of 1e-5, a global batch size of 192, and a maximum sequence length of 32,768 tokens, with DeepSpeed ZeRO-3 and BF16 precision.

Q2: "What is the reasoning effort used in GPT-6 Astra evaluation?"

We evaluate GPT-6 Astra with reasoning effort set to low and max_output_tokens=4096.

Q3. General Capability Retention

Reviewer: "Will this post-training process cause catastrophic forgetting of the original model's capabilities beyond spatial cognition?"

Thank you for this interesting question. We have additionally evaluated our models on TextVQA, MMMU, and MMBench to assess the impact of spatial post-training on general VLM capabilities.

ModelTextVQA(val)MMMU (val)MMBench (dev)
InternVL3-8B (Base)82.1555.8986.25
URUQI-Syn-8B80.7553.8984.79
URUQI-Mix-8B81.2353.1186.17
SenseNova-SI-1.574.9751.8982.99
URUQI-SI-Mix-8B75.1951.2283.08

The results suggest that spatial understanding can be improved while largely preserving general VLM capabilities. Specialized post-training may cause modest performance decreases, and incorporating more general-purpose data during post-training could help mitigate them. More broadly, the strong performance of GPT-6 Astra suggests that coherent spatial understanding and general capabilities can develop together, making spatial cognition an important direction for future VLM research.

AuthorsOpenReview ↗

Response to Reviewer XnT7 (Part 2)

Q4. Figure Organization

Reviewer: "I think Figure 2 should be the first figure of the paper. The current Figure 1 is actually making the paper harder to understand."

Thank you for this helpful suggestion. We agree that the motivation behind Figure 1 could be presented more clearly. We have revised Figure 1 to explicitly highlight two key phenomena: cross-view inconsistency and query-order sensitivity. We also clarify the M1/M2 queries, spatial predictions, query orders, and final QA outcomes to make the figure easier to follow.

We have also improved the accompanying explanation in the Introduction and figure caption. We first explain how Figure 1 examines the accuracy and consistency of self-motion and object-location predictions beyond final QA correctness, then describe the two observed phenomena, including GPT-5.5's query-order sensitivity. This establishes why correct spatial QA answers do not necessarily imply coherent spatial understanding, naturally motivating the interleaved episodic supervision introduced in Figure 2.

Thanks for the additional ablation experiments and explanations. I have raised the evaluation.

Dear Reviewer XnT7,

Thank you for the positive reassessment. We greatly appreciate your follow-up comment and are glad that our clarifications were helpful.

Best regards,

Submission7770 Authors

Reviewer_WB5ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper generates 11,738 motif-driven camera trajectories in OmniGibson scenes and compiles each into a multi-turn episode. Questions on self-motion (M1), persistent object mapping (M2) and spatial operations (M3) are interleaved at the points where their evidence becomes available. Fine-tuning InternVL3-8B on the resulting 611K QA raises the in-house benchmark from 15.8% to 47.7%. The model also improves on VSI-Bench, MMSI-Bench and MindCube, and a dense-trajectory evaluation measures pose and object-location errors.

Critical strengths
  • The M1 → M2 → M3 decomposition is clean, and aligning each question with the moment its evidence appears is a good design.
  • The model transfers to external benchmarks without using their training data, and test scenes are disjoint from training scenes.
  • The dense tracking analysis is informative, especially the pose-propagation study, which shows that initial estimates are good and that drift in estimated self-motion is the bottleneck.
Critical weaknesses
  • The paper never tests its core claim. There is no ablation comparing interleaved shared-context episodes with the same QA given as isolated single-turn samples, and none removing M1, M2 or M3. Without these, the claim that coupling motion and state updates is what matters is unverified.
  • External transfer is modest and mixed. The synthetic-only model drops below its base on VSI-Bench numerical tasks (48.0 → 39.3) and MindCube "around" (51.6 → 36.8), and stays below Qwen3-VL-8B on VSI-Bench. The best variant starts from SenseNova-SI-1.5 and adds only +0.6 / +2.7 / +0.6. The obvious control (the same 100K SenseNova data without URUQI data) is missing, so the Mix gains cannot be attributed to URUQI.
  • Diversity is limited: 38 training scenes from one simulator, which sits poorly with the "broad range of 3D scenes" and "scalable" framing. The large in-house gain is largely in-distribution.
  • The multi-turn premise is weakened by the paper's own result. Feeding the model its own earlier answers lowers localization accuracy (49.6 → 39.6), which suggests error accumulation or exposure bias from training on ground-truth history. Very low baseline scores (Spatial-MLLM at 5.0) also suggest answer-format mismatch on the in-house benchmark.
Questions for authors
  1. How does the model perform when trained on the same QA without episodes, and with each of M1/M2/M3 removed?
  2. How does SenseNova-SI-1.5 (and InternVL3-8B) perform when fine-tuned on the 100K SenseNova subset alone?
  3. How are answers parsed and scored on the URUQI benchmark, and what tolerance is used for numerical answers?
4 comments
AuthorsOpenReview ↗

Response to Reviewer WB5Z (Part 1)

Thank you for your valuable feedback and for recognizing the strengths of our supervision design, external generalization, and tracking analysis. We address the remaining concerns with additional ablations and clarifications below.

W1 & Q1. Contribution of Episodic Organization and Spatial Supervision

"There is no ablation comparing interleaved shared-context episodes with the same QA given as isolated single-turn samples, and none removing M1, M2 or M3."

Thank you for your suggestion. We have added two sets of experiments: a same-QA single-turn comparison to evaluate episodic organization, and M3-only, w/o M1, and w/o M2 ablations to examine the contributions of spatial supervision in Table R1 below:

Training ConfigurationURUQIVSI-BenchMMSI-BenchMindCube-tiny
M3-only29.0946.5831.1044.38
M2+M3 (w/o M1)35.9846.3030.2041.52
M1+M3 (w/o M2)45.2345.5829.9045.43
M1+M2+M3 (Single-turn)44.2846.9631.2042.29
URUQI-Syn-8B (Interleaved Episode)47.7347.2633.1045.90

(1) Episodic organization provides additional benefits beyond the isolated single-turn QA training. With identical M1–M3 questions and visual evidence, interleaved training consistently outperforms isolated single-turn training across all four benchmarks. This supports our central motivation that learning spatial supervision within continuous visual experiences provides benefits beyond treating the same questions independently.

(2) Full supervision achieves the best performance across all four benchmarks. More importantly, on external benchmarks, adding M1 or M2 individually does not always improve performance, whereas combining them yields better results. This suggests that the gains come not simply from adding more spatial QA, but from jointly learning self-motion, persistent object mapping, and spatial operations as interleaved capabilities.

We have incorporated these ablations and the single-turn comparison into Section 4.4 and Table 3 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer WB5Z (Part 2)

W2 & Q2. External Transfer and SenseNova-only Control

"External transfer is modest and mixed. The synthetic-only model drops below its base on VSI-Bench numerical tasks ... and MindCube 'around' ... The obvious control (the same 100K SenseNova data without URUQI data) is missing."

We would like to clarify that synthetic-to-real transfer and gains over an already spatially trained model are different comparisons. More importantly, our newly added SenseNova-100k-only ablation shows that adding URUQI data yields substantial additional gains across all three external benchmarks (+4.13/+9.80/+9.05) (Table R2). We explain these gains and the task-specific decreases below.

(1) Synthetic-only training shows substantial cross-domain transfer. Starting from InternVL3-8B, URUQI-Syn-8B improves VSI-Bench, MMSI-Bench, and MindCube-tiny by +5.12, +5.30, and +7.71 points, respectively. As shown in Table 2, the gains are particularly strong on VSI multiple-choice (+18.97), MindCube rotation (+11.00), and among (+16.00). These improvements come from only 38 synthetic training scenes, without using training data of targeted benchmarks.

(2) Task-specific decreases reflect complementary supervision needs. We acknowledge the decreases in VSI numerical and MindCube around with URUQI-Syn-8B. Our supervision focuses on maintaining coherent spatial understanding as the observer moves, rather than optimizing for specific benchmark questions. While absolute metric estimation can often benefit from single-view, metric-grounded supervision, URUQI-Syn-8B substantially improves spatial relations and transformations, including VSI multiple-choice (+18.97), MindCube rotation (+11.00), and among (+16.00).

(3) Crucially, these decreases are not an inherent trade-off of learning from visual experiences. Adding just 100k SenseNova examples raises VSI numerical from 39.28 to 55.38 and MindCube around from 36.80 to 57.20. Moreover, URUQI-Mix-8B consistently outperforms both the base model and our newly added SenseNova-100k-only baseline across all three external benchmarks (Table R2).

Training DataURUQIVSI-BenchMMSI-BenchMindCube-tiny
SenseNova-100k only17.8653.4330.0046.95
URUQI data + SenseNova-100k48.35(+30.49)57.56 (+4.13)39.80 (+9.80)56.00 (+9.05)

(4) We would like to clarify that the marginal gains over SenseNova-SI-1.5 are achieved on an already strong, benchmark-aligned baseline. SenseNova-SI-1.5 already benefits from extensive spatial training, including VSI-590K and MindCube training data, achieving 67.21% on VSI-Bench and 92.00% on MindCube-tiny. Despite this strong benchmark-specific training foundation, URUQI-SI-Mix further improves all three external benchmarks (+0.55/+2.70/+0.57).

More importantly, strong spatial QA performance does not necessarily imply coherent spatial understanding. Despite achieving 92.00% on MindCube-tiny, SenseNova-SI-1.5 scores only 17.02% on URUQI, which our training raises to 50.41% (+33.39). URUQI not only reveals this previously underexplored gap, but also enables models to maintain coherent spatial understanding while improving their performance on existing spatial QA benchmarks.

We have added the SenseNova-100k-only ablation to Table 2(b) and discussed its comparison with URUQI-Mix-8B in Section 4.3.

W4. Error Propagation and Persistent Spatial Tracking

Reviewer: "The multi-turn premise is weakened by the paper's own result. Feeding the model its own earlier answers lowers localization accuracy (49.6 → 39.6), which suggests error accumulation or exposure bias from training on ground-truth history."

Our dense tracking evaluation is designed to measure whether VLMs can maintain coherent spatial understanding as the observer moves. It reveals a substantial capability gap in existing VLMs. We agree that our model is affected by historical prediction errors, as localization accuracy decreases from 49.56% to 39.62%. However, even under this setting, URUQI-Syn-8B substantially outperforms Qwen3.8-27B (39.62% vs. 9.41%, as shown in Table 4).

This demonstrates that URUQI has developed meaningful persistent spatial tracking capabilities that existing VLMs largely lack, while revealing error accumulation as a key remaining challenge. We hope this finding encourages the community to develop more reliable spatial understanding in VLMs.

AuthorsOpenReview ↗

Response to Reviewer WB5Z (Part 3)

W3. Scene Diversity, Scalability, and the Benchmark Contribution

"Diversity is limited: 38 training scenes from one simulator, which sits poorly with the 'broad range of 3D scenes' and 'scalable' framing. The large in-house gain is largely in-distribution."

Thank you for raising this concern. We have revised “a broad range of 3D scenes” to “50 3D scenes” for precision. Importantly, using only 38 simulated scenes for training highlights the data efficiency of our approach. The resulting visual-experience supervision achieves substantial gains on three external real-world benchmarks without using their training data (as shown in Table 2). These transfer gains suggest that diverse spatial supervision can arise from how an observer moves, tracks objects, and updates spatial relationships, rather than relying primarily on variations in scene appearance.

Regarding scalability, our key insight is that a single scene can support diverse visual experiences and spatial supervision. Using 7 spatial motifs, we generate 8,818 training trajectories with different movement paths, observation sequences, object combinations, and visibility changes. These experiences provide diverse supervision for spatial understanding, while our reusable motif-driven pipeline can naturally extend to more diverse 3D environments.

Regarding in-distribution gains, we wish to clarify that URUQI evaluates a distinct spatial capability, not simply QA performance on simulated scenes. Rather than evaluating isolated spatial questions, URUQI uses temporally connected queries to measure whether models maintain coherent spatial understanding as the observer moves. Despite achieving 92.00% on MindCube-tiny, SenseNova-SI-1.5 scores only 17.02% on URUQI, whereas GPT-6 Astra reaches 50.08% without URUQI training (Table 1). This reveals a spatial capability gap that previous QA benchmarks do not adequately capture.

We have further clarified these contributions in the revision, highlighting that diverse visual experiences from limited scenes can develop coherent spatial understanding and transfer to real-world tasks.

Q3 & W4. Answer Parsing and Evaluation Fairness

W4: "Very low baseline scores (Spatial-MLLM at 5.0) also suggest answer-format mismatch on the in-house benchmark."

Q3: "How are answers parsed and scored on the URUQI benchmark, and what tolerance is used for numerical answers?"

We examined all 52,920 Spatial-MLLM responses, together with their prompts and conversation histories. The dominant failure was missing answer content: 40.36% of all responses provided option letters to open-ended questions. Representative examples are shown below.

QuestionFormat example provided in the inputModel output
“Describe your movement between Images 1 and 2.”“Move 1.20 m forward, 0.40 m left. Turn right 30 degrees.”<answer>A</answer>
“Which frame was the bookcase … last seen in?”3<answer>B</answer>
“How far is the bookcase … in the forward/backward and left/right directions?”“1.20 m forward, 0.40 m left.”<answer>1.5</answer>

We also found malformed answer formats. Only 3.06% of all responses contained correct answers that could be recovered by relaxing the parsing of malformed wrappers (e.g., <>B</answer>). We expect spatially capable VLMs to follow basic formatting instructions, especially when explicit examples are provided. Such failures are therefore counted as incorrect. These findings indicate that the low score primarily reflects failures to provide the requested answer content and follow explicit instructions, rather than parsing limitations.

Regarding answer parsing and scoring, our evaluation follows a two-stage pipeline. We first apply deterministic parsing, using Qwen3.8-Max only as a fallback extractor for open-ended responses, without judging correctness. A deterministic scorer then applies predefined scoring criteria. Discrete answers require exact matching. Continuous predictions use scale-dependent tolerances of 0.5–2.0 m for distances and positions, 0.5 m per axis for self-motion translations, 10° for angles, and 5% of image width for point correspondences. Multi-component answers must satisfy all required criteria, while missing, incomplete, or ambiguous responses are counted as incorrect. Full scoring rules and question-specific tolerances are provided in Appendix D.

We have clarified the evaluation pipeline in Section 4.1 and provided the details in Appendix D.

Dear Reviewer WB5Z,

As the discussion period is coming to an end, we would greatly appreciate your feedback on whether our clarifications have addressed your main concerns.

Thank you again for your time and consideration.

Best regards,

Submission 7770 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.