acceptodds
Withdrawn from ICLR 2027

World SLAM Model: Joint World Modeling for SLAM and Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce World SLAM Model (WSM), a unified framework that brings the SLAM paradigm directly into downstream navigation. Rather than treating SLAM merely as an upstream module that provides poses, maps or tokens, WSM adopts its core mechanisms, including incremental state updates with persistent memory and backend refinement of accumulated errors, to maintain a consistent world state during interaction. Given the current observation and a navigation goal, WSM predicts future visual states and jointly estimates their camera motion and dense geometry, grounding visual prediction in an evolving spatial world state. This spatial state is continuously updated as new observations arrive and provides the basis for action generation and closed-loop navigation. WSM is trained end-to-end with a joint navigation–SLAM objective, enabling downstream navigation to benefit directly from SLAM-style state maintenance and refinement while preserving accurate geometric estimation. Experiments demonstrate improved navigation performance together with strong SLAM accuracy, highlighting the potential of SLAM as an intrinsic mechanism for long-horizon world modeling and embodied interaction.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Model, SLAM, Navigation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tcncOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces World SLAM Model, an approach that unifies SLAM and navigation, by bridging world modeling of future states with traditional SLAM methods. In a nutshell, the approach uses video generation to predict (dream) future frames towards a goal and then SLAM processes these frames to produce future poses and geometry (essentially working as an inverse dynamic model). These poses are also used for RL-based post training. The core of the approach uses a Mixture of Transformers to jointly train future prediction (generation expert) and geometric prediction (SLAM expert). The approach is evaluated on a navigation benchmark and a SLAM benchmark.

Critical strengths
  • Connecting world models and SLAM systems is a reasonable and seemingly technically sound approach that could be expected to improve navigation performance.
  • Leveraging the reconstructed geometry for RL self-training seems a clear advantage of the proposed framework.
  • The approach is shown to outperform baselines in experiments, and the ablations support the design choices.
Critical weaknesses
  • Using the term “SLAM” in this context seems misleading since the approach does not retain a persistent memory and does not perform loop closure. Technically, this provides a long context, but it cannot maintain the full history of information throughout execution. What happens if we gather more than 128 latent frames? How does performance change as the scene gets larger? The experiments seem to focus on relatively short trajectories (Figure 4).
  • The SLAM evaluation is not comprehensive enough. The authors should examine other SLAM methods, such as VGGT-Omega, ORB-SLAM, and DROID-SLAM, and evaluate on datasets such as TUM, Replica, 7-Scenes, KITTI, or similar if they are claiming a contribution to SLAM.
  • The paper does not fully justify why "joint modeling" or "joint training" is necessary or why it should be considered a contribution. I suggest comparing against a model variant where the generation expert and SLAM expert are not jointly trained but are connected in a modular manner. For example, you could train the video generation model to condition on pose and depth, and use a state-of-the-art dense 3D prediction model, such as VGGT to estimate pose and depth. If your jointly trained model performs better than a separately trained but serially connected model, then the ablation would better justify the proposed design.
  • Limited navigation benchmarks. InternNav includes two datasets (https://github.com/InternRobotics/InternNav). The paper reports performance on InternScene, but where are the results for the ClutteredEnv dataset? Since both datasets are under the same codebase, I assume it should be easy to evaluate on the latter. For more comprehensive experiments, the authors should also consider other simulation environments, such as Habitat and Isaac Sim. Only 40 scenes from InternScene do not provide sufficient evidence of the performance of the proposed approach.
  • Navigation baselines. The paper presents the use of a world model as a core module for planning as one of its contributions. However, the chosen baselines are all planners without a world modeling component. This raises concerns about whether the proposed method is actually at the state-of-the-art in world-model-based navigation. The authors should consider comparing against Navigation World Models [1]. To make a convincing argument that "joint modeling" contributes to performance, the authors should consider comparisons against modular navigation methods with a world modeling or future imagination component, such as STRIVE [2], SGImagineNav [3], and ImagineNav [4]. In addition, the future prediction quality is not evaluated. How does the quality of predicted frames affect pose and depth estimation? How does it affect navigation performance?
  • No video attachment showing a full navigation episode. The paper only shows the final results and snapshots of navigation. However, it is difficult to be convinced that the method works smoothly without a full video showing a successful navigation episode. In addition, the dreamed visuals seem to be very short-horizon. With such local predictions, how can the system avoid following suboptimal trajectories and getting stuck in places such as tight corners?

[1] Bar, Amir, et al. "Navigation World Models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025 [2] Zhu, Hao-Kun et al. “STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation.” ArXiv abs/2505.06729 (2025) [3] Y. Hu et al., "Imaginative World Modeling With Scene Graphs via VLM for Object Navigation," in IEEE Robotics and Automation Letters, vol. 11, no. 11, pp. 12855-12862, Nov. 2026, doi: 10.1109/LRA.2026.3732899. [4] Zhao, Xinxin, et al. "ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imagination." International Conference on Learning Representations, 2025

Questions for authors
  • The paper does not fully justify why "joint modeling" or "joint training" is necessary or why it should be considered a contribution. How does the proposed approach compare against a model variant where the generation expert and SLAM expert are not jointly trained but are connected in a modular manner?
  • What are the runtime and memory requirements of the model? Although the paper mentions that inference runs on an RTX 4090 GPU, providing runtime measurements would offer useful insights into practical deployment.
Reviewer_E3BZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes World SLAM Model (WSM), a framework that unifies goal-conditioned video generation, depth estimation, and camera pose estimation within a single architecture for monocular RGB navigation. Its main contribution is to incorporate the SLAM mechanism itself directly into world modeling, rather than merely consuming outputs from an external SLAM module, such as poses, maps, or tokens, for downstream navigation. The same SLAM expert processes both real observations and dreamed future observations to estimate current world states and recover future camera trajectories, respectively. Through a persistent KV cache and local backend refinement, WSM integrates state maintenance and error correction into the navigation process. The authors evaluate closed-loop navigation on InternVLA-N1 and camera pose estimation and dense reconstruction on held-out InternVLN-N1 trajectories, with ablation studies examining the SLAM expert, the local backend, and post-training.

Critical strengths
  1. A technically motivated framework integrating spatial state maintenance with generative navigation. WSM shares a SLAM expert across observed and generative sequences, using persistent memory and local backend refinement to connect localization, planning, and execution feedback. This provides a promising integration, although the shared mechanism’s independent contribution requires further validation.

  2. Improved closed-loop navigation success and path efficiency. Across 4,000 episodes in 40 unseen indoor simulated environments, RGB-only WSM achieves 70.3%/73.9% Home/Commercial success rates, exceeding re-evaluated LoGoPlanner by 10.7/8.1 percentage points, with higher SPL. These consistent gains provide strong empirical support within the evaluated setting.

Critical weaknesses
  1. The core mechanism lacks sufficient analysis and experimental evidence. The paper argues that geometric and navigation objectives jointly shape the shared world-state representation and its maintenance mechanisms, but provides limited targeted analysis or controlled experiments supporting this claim. The experiments demonstrate the effectiveness of the complete system but do not isolate the benefits of integration: the IDM ablation also removes depth estimation and localization of observed frames, while changing the historical state conditioning.
  2. The specific fusion design is insufficiently justified. WSM uses the same SLAM expert for observed and dreamed sequences and joint self-attention within a Mixture-of-Transformers (MoT) architecture. However, the paper does not sufficiently explain the rationale for these choices or demonstrate their advantages through controlled comparisons with reasonable alternatives.
Questions for authors
  1. The SLAM IDM ablation simultaneously removes depth estimation, localization of observed frames, and corresponding historical state conditioning. How do the authors distinguish gains from additional geometric supervision from those of the shared SLAM expert and joint state maintenance?
  2. What motivates using the same SLAM expert for observed and dreamed sequences and joint self-attention within the Mixture-of-Transformers (MoT) architecture? Could the authors provide evidence that these choices offer advantages over reasonable alternatives, such as separate experts or conditioning on geometric features?
Reviewer_8xtLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WSM combines visual prediction and spatial estimation for RGB-only point-goal navigation. A generation expert predicts future images toward a supplied goal position, while a shared SLAM expert estimates depth and camera poses from both real and generated images. The robot follows part of the predicted trajectory, updates its spatial history from new observations, and replans. Experiments report improved navigation performance and promising pose-estimation accuracy.

Critical strengths
  1. An interesting shared prediction–estimation framework. Using the same spatial estimator for real and imagined observations connects the robot’s current understanding with its planned movement. Updating the observed spatial history and refining recent estimates provides a coherent way to support repeated prediction and execution.

  2. Substantial navigation evaluation with encouraging results. The benchmark covers 4,000 episodes across 40 environments reported as disjoint from training. WSM achieves 70.3% and 73.9% success in Home and Commercial settings using RGB alone, exceeding the re-evaluated LoGoPlanner baseline. The backend and post-training ablations provide useful additional evidence.

Critical weaknesses
  1. The comparisons do not isolate the central benefit of integrating SLAM and navigation. The main ablation removes depth estimation and localization of observed frames together. Its large performance drop shows that these capabilities matter, but does not establish that the proposed joint architecture is better than a comparable modular system. Differences in model capacity, pretraining, and navigation post-training further complicate attribution of the gains over existing baselines. Post-training alone contributes roughly 5–6 success-rate points. A controlled comparison with a separate geometry estimator and a comparable navigation model would better support the main claim; a direct navigation-world-model baseline is also missing.

  2. The evaluation provides limited evidence for long-horizon metric consistency and practical deployment. Backend refinement is restricted to the current chunk, with older chunks left fixed, and the paper does not demonstrate loop closure or correction of older map estimates. The SLAM comparison uses only SLAM-Former, measures trajectory error after scale-inclusive alignment, and reports worse reconstruction accuracy than its fine-tuned baseline. Thus, the low aligned pose error alone does not establish reliable metric scale during navigation. The navigation tests are entirely simulated, planning latency is unreported, and the held-out SLAM split is not clearly stated to contain unseen scenes.

Questions for authors
  1. Can the authors provide a controlled comparison against a modular geometry-estimation and navigation system with similar data, capacity, and training budget?
  2. What is the exact composition and size of the training dataset? Does the 5% SLAM test split hold out entire scenes or only trajectories?
  3. What are the pose errors without scale correction, and how does localization drift change with trajectory length and backend refinement?
  4. What is the end-to-end planning latency, including image generation and trajectory recovery, and how does memory usage grow with episode length?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.