- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WorldMind, a framework for test-time scaling of visual spatial reasoning through latent world simulation. The framework selects latent features from a world model and aligns them with the VLM’s embedding space using a lightweight adapter. WorldMind uses supervised fine-tuning on simulated reasoning trajectories, followed by process-verifiable reinforcement learning that further refines the policy using progress rewards gated by action and state validity, alongside correctness and format rewards. Experiments show that WorldMind, built on Qwen3-VL-8B and Stable Virtual Camera, outperforms the evaluated baselines on VSI-Bench, MMSI-Bench, and MindCube-Tiny.
Critical strengths
-
Direct interaction through world-model latents. While existing approaches couple VLMs with world models for active visual imagination primarily through rendered RGB images, WorldMind directly incorporates intermediate world-model representations into VLM reasoning, providing an alternative interface for spatial reasoning.
-
Informative ablation studies. The paper examines the effects of latent world simulation, latent feature selection, and reward design. In the interface ablation, latent simulation outperforms RGB simulation by 1.8, 2.6, and 0.1 on VSI-Bench, MMSI-Bench, and MindCube, respectively.
Critical weaknesses
-
The paper does not establish how performance scales with additional test-time computation. Although “test-time scaling” is emphasized in both the title and the motivation, the experiments do not report accuracy as a function of the simulation budget, the actual number of world-model calls, or inference time. To the best of my knowledge, the main technical contribution appears to lie primarily at training time.
-
Limited novelty and insufficient baseline comparisons. The main distinction of this work is to incorporate intermediate world-model representations directly into VLM reasoning instead of using synthesized RGB images. However, the ablation study shows only modest gains from this substitution, leaving the significance of its incremental contribution unclear. The evaluation would benefit from comparisons with stronger and more recent baselines on VSI-Bench, MMSI-Bench, and MindCube.
Questions for authors
Please see the weaknesses above.
5 comments
In the following, please find our new experiment results and responses to the reviewer's concern of
Limited novelty and insufficient baseline comparisons. The ablation study shows only modest gains from this substitution, leaving the significance of its incremental contribution unclear.
We repeated the ablation in Table 3 with a larger training dataset, using approximately 25% of the full-scale training budget, as conducting multiple full-scale experiments remains infeasible within the rebuttal period due to computational and time constraints. The additional results are presented below:
| Method | VSI-Bench | MMSI-Bench | MindCube |
|---|---|---|---|
| Base VLM | 57.4 | 29.4 | 40.9 |
| + RGB Simulation | 65.6 | 35.1 | 50.2 |
| + Latent Simulation | 70.8 | 41.7 | 57.4 |
The results show that the advantage of latent simulation becomes more pronounced with increased training, outperforming RGB simulation by 5.2, 6.6, and 7.2 percentage points on VSI-Bench, MMSI-Bench, and MindCube, respectively. These improvements are larger than those reported in the original small-scale ablation, further supporting the effectiveness of the latent simulation interface.
Furthermore, we emphasize that WorldMind's contributions extend beyond replacing synthesized RGB images with intermediate world-model representations. Another key contribution is our process-verifiable RL framework, which improves multi-step spatial reasoning by jointly considering reasoning progress, action validity, and simulation reliability. Since the RGB Simulation variant also benefits from our RL framework, the performance gap between RGB and latent simulation reflects only the additional benefit of the latent interface, rather than the full contribution of WorldMind. The main results therefore demonstrate the combined effectiveness of our latent simulation interface and RL-based reasoning framework.
Dear reviewer,
As the rebuttal period is approach to the end, we would like to kindly check if our rebuttal has sufficiently addressed all your concerns and questions. Please feel free to let us know if any further clarifications, explanations or experiments are needed, and we will be more than happy to discuss further in any aspect.
Again, thank you so much for all your attention and efforts!
We thank the reviewer for the constructive comments. Please find our responses to the reviewer's concerns below.
To address this concern, we conducted an experiment by enforcing a specific number of latent simulation steps from 0 to 10. Each latent simulation step corresponds to one world-model call. As shown below, we evaluate WorldMind's accuracy under different latent simulation budgets across four task categories in VSI-Bench. The results demonstrate that the effectiveness of additional latent simulation varies across tasks. Tasks requiring substantial mental spatial transformations, such as Relative Direction (Rel. Dir) and Route Planning (Route Plan.), benefit from larger simulation budgets, with accuracy improving as the number of latent steps increases. In contrast, tasks that rely less on spatial simulation, such as Object Counting (Obj. Count) and Appearance Order (App. Order), achieve their highest accuracy with only one latent simulation step, after which additional simulation may introduce redundant information.
These findings highlight the importance of task-adaptive test-time simulation and scaling rather than uniformly increasing the inference budget. In practice, WorldMind learns to adaptively determine the number of simulation steps through RL, and the task-specific accuracies reported in Table 1 are close to the corresponding best results under different simulation budgets in the table above.
Regarding the modest gains in Table 3, as noted in Line 465, these results were obtained from small-scale experiments with small amount of training data, due to the computational cost of conducting multiple full-scale training runs under different settings. Therefore, they do not fully reflect the potential benefits of latent simulation at larger training scales. As shown in Figure 3(a), RGB-based inputs initially outperform latent representations with limited training data, since pretrained VLMs are already well aligned with the image modality. However, as the amount of fine-tuning data increases, latent-based representations catch up with and eventually surpass their RGB counterparts. To further validate this trend, we repeated the ablation in Table 3 with a larger training dataset (but still not full-scale experiment as it’s infeasible within the rebuttal period due to computational and time constraints) and we will report the experiment results later in a separate comment.
Further, Table 3 reports average accuracy across different tasks, which may obscure larger improvements in specific task categories. To provide a more detailed analysis, we report the per-task accuracy on VSI-Bench for the models evaluated in Table 3, as shown in the table below. The results reveal that latent simulation provides more substantial improvements on tasks requiring mental spatial transformations, particularly Relative Direction and Route Planning, where the performance gap between latent-based and RGB-based simulation is more pronounced.
We explained our baseline selection criteria in Appendix A.1. We include methods that (1) report sufficiently complete results on the evaluated benchmarks, (2) release checkpoints that can be evaluated on the remaining benchmarks, or (3) provide implementations that enable reliable reproduction under the official evaluation protocols. We prioritize strong and recent methods with comparable model sizes (approximately 7–8B parameters), while relaxing these requirements for world-model-based approaches to ensure representative comparisons. Accordingly, we include competitive baselines such as Cambrian-S-7B, a recent method achieving strong performance on VSI-Bench, and SpatialDreamer, which is closely related to our approach.
We also considered more recent methods, such as ConsiSpace and UC-SSRL. However, we were unable to reproduce results comparable to those reported in their papers under our evaluation settings. Hence, we excluded them to maintain the reliability and fairness of comparisons. If needed, we will incorporate results on ConsiSpace and UC-SSRL into the paper if got accepted.