- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents WorldWeave, a world-modeling framework that jointly learns predictive dynamics across vision-and-language navigation (VLN) and embodied visual tracking (EVT). The proposed approach learns task-agnostic latent actions from unlabeled videos through inverse and forward dynamics modeling, enabling dynamics sharing across heterogeneous action spaces. The learned future-state predictions are then incorporated into downstream policies through task-specific adaptation modules. Experiments on multiple simulated benchmarks and real-world robot deployments demonstrate improvements over task-specific baselines.
Critical strengths
Meaningful research motivation. Learning shared predictive dynamics across different embodied tasks is an interesting research direction. VLN and EVT provide complementary settings involving ego-motion and target motion, making them reasonable tasks for studying cross-task dynamics learning. Practical framework design. Learning latent transitions without requiring unified action spaces is a reasonable approach for various embodied tasks. The combination of shared dynamics and task-specific adaptation offers a practical way to mitigate negative transfer during multi-task learning. Extensive experimental evaluation. The paper evaluates WorldWeave across multiple VLN and EVT benchmarks, including R2R-CE, RxR-CE, EVTBench, UnrealZoo, and Gym-UnrealCV. The consistent improvements over matched baselines are encouraging.
Critical weaknesses
Limited methodological novelty. Most technical components are closely related to existing methods. Learning latent actions from video transitions has been extensively explored in prior work, including Genie [1] and LAPA [2]. Predictive world modeling has also been investigated separately for VLN (NavMorph [3]) and EVT (USS [4]). Furthermore, sharing world dynamics while introducing task-specific components has been explored in multi-task world modeling, such as Mixture-of-World Models [5]. Although applying these ideas jointly to VLN and EVT provides a different task setting, the proposed framework appears largely to combine established techniques. Insufficient evidence for the benefit of cross-task dynamics sharing. The main contribution of the paper is the claim that jointly learned world dynamics can benefit multiple embodied tasks. However, there lacks a comparison against independently pretrained world models for VLN and EVT under matched data and computational budgets. Therefore, it remains unclear whether sharing the dynamics model provides positive knowledge transfer compared with simply training separate dynamics models using the same data. Lack of direct validation of learned world dynamics. The paper primarily evaluates downstream task performance rather than the quality of the learned dynamics. There is no direct quantitative evaluation of future-state prediction accuracy, latent-action consistency, or cross-task dynamics transfer. Moreover, sharing the parameters of a dynamics model does not necessarily imply that the model learns a unified or transferable transition representation. Since the latent actions are inferred from observation changes rather than grounded in physical actions, they may also encode task-specific visual changes without capturing reusable motion dynamics. Additional analyses are needed to demonstrate what information is shared and whether the learned transition representations genuinely transfer across tasks. References: [1] Bruce et al., Genie: Generative Interactive Environments, ICML 2024. [2] Ye et al., Latent Action Pretraining from Videos, ICLR 2025. [3] Yao et al., NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments, ICCV 2025. [4] Xie et al., USS: Unified Spatial-Semantic Prompts for Embodied Visual Tracking with Latent Dynamics Learning, 2026. [5] Zhang et al., Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamics, ICLR 2026.
Questions for authors
Could the authors clarify the key novelty of WorldWeave compared with existing latent-action learning and multi-task world-modeling approaches? In particular, what is the fundamental contribution beyond combining established techniques in a unified VLN-EVT framework? What is the direct benefit of sharing world dynamics across VLN and EVT? Could the authors compare the proposed shared world model against two independently pretrained task-specific world models, using matched pretraining data, model capacity, and computational budgets? This comparison seems essential to validate the central hypothesis. What evidence supports the transferability of the learned latent actions and dynamics representations? Could the authors provide direct evaluations of future-state prediction accuracy or cross-task transfer experiments? It would be particularly interesting to investigate whether latent actions corresponding to similar ego-motion patterns are consistent across VLN and EVT. How are the task-specific state representations kept compatible with the frozen dynamics model? Since Stage II introduces separate task-specific LoRA adapters while freezing the pretrained dynamics modules, it would be helpful to clarify whether the resulting representation shifts affect prediction accuracy.