- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies spatial mental modeling for multi-image spatial reasoning. Instead of invoking external 3D tools at inference or relying on dense spatial annotations, it aligns latent states with compact camera and scene tokens extracted from a pretrained VGGT-Ω model. Stage 1 combines MSE-based geometric alignment with next-token prediction. Stage 2 augments GRPO with latent-mode entry exploration. The default backbone is Qwen2.5-VL-3B. The training set contains 15K examples sampled from MindCube-Train and SPAR-7M, including 10K for SFT and 5K for RL, with reasoning traces generated by Qwen3.5-122B. Across three multi-image benchmarks, the method improves the overall score from 29.4 to 55.6 over Qwen2.5-VL-3B and exceeds HATCH-3B by 12.0 points. It also improves performance on the single-image SPAR-Bench-SI and CV-Bench.
Critical strengths
- The controlled comparison with different methods provides the clearest evidence for the method.
- The ablation of latent-entry exploration controls for the additional rollout budget. When failed prompts receive the same number of additional standard rollouts without entry-position bias, performance on MindCube-Tiny is 3.8 points lower. Removing either camera or scene tokens reduces performance by 2.8 and 4.4 points, respectively, while removing the entire latent segment causes a 14.9-point drop. These ablations are well aligned with the proposed components.
Critical weaknesses
- The overall framework is highly similar to 3DThinker. Both methods align latent states with geometry extracted by a pretrained 3D foundation model, combine geometry alignment with next-token prediction during SFT, and subsequently apply GRPO. The novelty and conceptual distinction from 3DThinker should therefore be articulated more carefully.
- The primary Qwen backbone is relatively dated. Although the paper evaluates different model families and scales, the main experiments are centered on Qwen2.5-VL. Validation on newer Qwen3.6/3.8, would better establish whether the method remains beneficial with stronger spatial representations and reasoning capabilities. This is consequential because improvements obtained on an older backbone may shrink as the underlying model becomes more capable.
I would like to raise my score if the author can solve my concerns.
Questions for authors
- Please report the overlap between the 15K training examples and MindCube-Tiny, MMSI-Bench, and SPAR-Bench at the image, scene, and question-template levels. If SPAR-Bench overlaps with SPAR-7M, or if MindCube-Tiny shares scenes with the MindCube training data, the main table cannot be interpreted as evidence of out-of-domain generalization.
- If HATCH and 3DThinker are trained using the same Qwen3.5-122B-generated reasoning traces as your method, while removing only the proposed geometric latent components, how much of the gain remains on MindCube-Tiny and MMSI-Bench? This comparison would help separate the contribution of the method from that of the teacher-generated data.
8 comments
Rebuttal by Authors [2/4]
W2. Validation on newer Qwen backbone.
Thank you for pointing this out. We agree that demonstrating applicability on newer backbones is important. Following your suggestion, we have conducted new experiments on Qwen3.8-27B across both multi-image and single-image spatial reasoning benchmarks. As shown in Table E, our method consistently improves over the base model on these benchmarks, indicating that it remains beneficial with a newer backbone. We have added these results to Appendix C.2 of the revised manuscript.
Table E: Results with the Qwen3.8-27B backbone on MindCube-Tiny, MMSI-Bench, SPAR-Bench-MV, SPAR-Bench-SI, and CV-Bench.
| Models | MindCube-Tiny | MMSI-Bench | SPAR-Bench-MV | SPAR-Bench-SI | CV-Bench |
|---|---|---|---|---|---|
| Qwen3.8-27B (Base) | 46.0 | 31.8 | 45.7 | 52.8 | 87.5 |
| Ours (SFT) | 84.5 | 36.6 | 55.8 | 57.5 | 89.0 |
| Ours (SFT+RL) | 88.4 | 40.5 | 63.2 | 61.0 | 90.3 |
Rebuttal by Authors [3/4]
Q1. Clarification on overlap between training samples and benchmarks.
Thank you for raising this question. Our 15K training samples consist of 5K from the SPAR-7M training split and the full MindCube-Train set (10K). When sampling from SPAR-7M, we selected only multi-image samples and excluded all scenes that appear in SPAR-Bench or in the ScanNet and Matterport3D portions of MMSI-Bench. The overlap at the image, scene, and question-template levels is summarized in Table F, and we have added the details to Appendix A.3 of the revised manuscript.
Table F: Overlap between our training samples and each benchmark.
| Benchmark | Image | Scene | Question template | Setting |
|---|---|---|---|---|
| MindCube-Tiny | No | Yes | Yes | In-domain |
| SPAR-Bench | No | No | Yes | In-domain |
| MMSI-Bench | No | No | No | Out-of-domain |
| CV-Bench | No | No | No | Out-of-domain |
- In-domain benchmarks. SPAR-Bench shares no images or scenes with our SPAR-7M samples, but shares question templates. MindCube-Tiny shares no images with MindCube-Train, but shares scenes and question templates. We regard both as in-domain benchmarks.
- Out-of-domain benchmarks. MMSI-Bench and CV-Bench share no images, scenes, or question templates with our training samples. MMSI-Bench aggregates images from eight sources (ScanNet, nuScenes, Matterport3D, Ego4D, AgiBot-World, DTU, DAVIS 2017, and Waymo). It requires the model to reason about domains absent from our training data, such as autonomous driving and robotic manipulation. CV-Bench is built from ADE20K, COCO, and three Omni3D subsets (Hypersim, SUN RGB-D, and nuScenes), covering diverse indoor/outdoor environments from datasets entirely distinct from our training sources.
Rebuttal by Authors [1/4]
Dear Reviewer zmSD,
Thank you for your constructive suggestions. We are glad that you found the controlled comparisons and ablations to provide clear evidence for our method. We sincerely hope our responses below could address your concerns.
W1. Novelty and conceptual distinction from 3DThinker.
Thank you for your valuable suggestion. We agree that articulating the distinction from 3DThinker more carefully would strengthen the paper. We have revised the manuscript in Sections 2.3 and 4.2 as you suggested. With the shared training framework (SFT+RL), our method differs from 3DThinker in the following respects:
We compare 3DThinker and our method at the SFT stage directly in Appendix C.1. With the same geometric teacher (VGGT-Ω), backbone, and training data, ours outperforms 3DThinker by 7.4 points on MindCube-Tiny (Figure 7). In addition, when 3DThinker is retrained on our 15K samples, our method outperforms it on all benchmarks at both stages (please see our response to Q2).