acceptodds
Under review as a conference paper at ICLR 2027

Building Robust World Models for VLA RL: Visual Generalization and Long-Horizon Consistency

Abstract

Vision-language-action (VLA) policies can be post-trained in learned simulators, but current world models often fail under visual shifts and accumulate errors during long-horizon rollouts. We present Sword, a style-robust world model for policy post-training. Sword addresses these failure modes with two complementary components. Structure-Guided Style Augmentation (SGSA) uses depth, segmentation, and task-conditioned style transfer to diversify visual appearance while preserving task-relevant geometry and semantics. Dynamic Latent Bootstrapping (DLB) reuses cached model-predicted latents as context during fixed-window training, exposing the model to self-generated inputs without requiring full-sequence rollouts. We further introduce LIBERO-Mixed, an evaluation set combining original episodes with style-transferred episodes generated using prompts held out from training. Experiments on LIBERO show that Sword improves prediction quality and temporal consistency over representative action-conditioned world models, including under held-out style shifts. In GRPO post-training, using Sword as the simulator also yields higher VLA task success rates than using WoVR.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Model

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_a4RiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Sword, an action-conditioned world model for VLA reinforcement learning, targeting visual generalization and long-horizon prediction consistency. It introduces Structure-Guided Style Augmentation (SGSA), which uses depth, segmentation, and task information to generate visually diverse training videos, and Dynamic Latent Bootstrapping (DLB), which incorporates cached model predictions as training contexts. The method is evaluated on LIBERO-Spatial through video prediction metrics, component ablations, and GRPO post-training of OpenVLA-OFT. The authors report improved prediction quality under original and held-out style conditions, along with higher downstream policy success rates than training with WoVR.

Critical strengths

None

Critical weaknesses
  1. The incremental contribution over prior work is insufficiently established, particularly for SGSA.
    Seeing Realism from Simulation: Efficient Video Transfer for Vision-Language-Action Data Augmentation already uses depth conditioning, task-related text, and Cosmos-Transfer 2.5 for robotic video augmentation, substantially overlapping with the main SGSA pipeline. The manuscript neither discusses this directly relevant work nor provides a controlled comparison demonstrating the additional value of its design choices, such as segmentation conditioning. Comparing augmentation against no augmentation establishes its usefulness, but does not establish an advance over existing augmentation methods. This weakens the positioning of SGSA as a major methodological contribution.

  2. The benchmark coverage is too limited to substantiate the generality of the claimed robustness and downstream utility.
    All experiments are confined to LIBERO-Spatial, a single task suite, without evaluation on other LIBERO suites, independent simulation benchmarks, or real robots. LIBERO-Raw and LIBERO-Mixed are evaluation settings derived from the same task source, differing primarily in the inclusion of synthetic visual changes; they do not constitute independent benchmark coverage. The reinforcement learning evaluation is similarly restricted to LIBERO-Spatial, using only OpenVLA-OFT with GRPO and comparing Sword against WoVR as the training environment. Policy post-training under style-transferred conditions remains future work. Consequently, the current evidence does not establish whether the reported benefits extend across different tasks, environments, and interaction dynamics. This narrow experimental scope is a substantial limitation for a paper motivated by robust world models for VLA reinforcement learning.

  3. The component analysis does not sufficiently isolate the value of key design choices or their contribution to downstream policy improvements.
    The paper includes a complete SGSA × DLB ablation and comparisons of eight cache configurations. However, these experiments primarily establish module-level effectiveness, without sufficiently explaining the contribution of the specific mechanisms. SGSA lacks controlled ablations isolating depth, segmentation, and task-text conditioning. DLB lacks direct comparisons, under the same backbone, against alternative strategies for mitigating the training–inference distribution mismatch. Moreover, the component ablations primarily report video prediction metrics, without establishing how each component contributes to downstream policy success. The current analysis therefore provides limited evidence attributing the final gains to the particular mechanisms emphasized in the paper.

Questions for authors
  1. Novelty relative to prior work: What are the substantive technical differences between SGSA and Seeing Realism from Simulation: Efficient Video Transfer for Vision-Language-Action Data Augmentation? Can a controlled comparison establish the additional benefit of SGSA’s segmentation conditioning over the prior depth-and-text-guided augmentation pipeline?

  2. Benchmark coverage: Can you provide results on an additional LIBERO task suite or an independent simulation benchmark? Since both LIBERO-Raw and LIBERO-Mixed derive from LIBERO-Spatial, such evidence would materially affect my assessment of whether the reported gains generalize beyond the current task distribution.

  3. RL evaluation protocol: Where are the policy success rates in Table 2 measured—in the original LIBERO simulator or in a learned world model? How are rewards, success, and termination determined during world-model-based training? Please also specify the number of seeds and evaluation trials, and report variability across runs.

  4. Component attribution: Do SGSA and DLB individually improve downstream policy success under a matched training budget? Can you provide a policy-level component ablation and a same-backbone comparison of DLB against a simpler strategy for mitigating the training–inference context mismatch? These comparisons would help establish which mechanisms account for the reported benefits.

Reviewer_Xe28OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose Sword, a style-robust world model for VLA post-training. Two main components include (1) Structure-Guided Style Augmentation (SGSA), using Cosmos-Transfer 2.5 with depth, segmentation, and task conditioning for style transfer, and (2) Dynamic Latent Bootstrapping (DLB), which reuses cached model-predicted latents during fixed-window training to mitigate exposure bias. Experiments show improvements over WoVR in world-model rollouts and downstream VLA performance.

Critical strengths
  • The paper is clearly written, allowing readers to quickly grasp the main idea.
  • The source code and GPU-hours details are provided, benefiting reproducibility.
Critical weaknesses
  • Limited novelty of SGSA. SGSA largely builds on existing Cosmos-Transfer2.5 capabilities, and is a common augmentation technique not specific to world models. Prior works like [1] also uses similar techniques for data augmentation. Should compare SGSA with these prior techniques to showcase the advantages.
  • Limited novelty of DLB. Exposure bias from teacher forcing is well known. Prior works like [2] and [3] already propose potential solutions. The paper should compare DLB with these existing techniques.
  • Why introduce LIBERO-Mixed? Existing benchmarks like LIBERO-Plus already evaluate OOD robustness. Since LIBERO-Mixed uses the same style-transfer pipeline as training, it is unclear how well the claimed robustness generalizes beyond this augmentation distribution, and may potentially favor SGSA.
  • Limited comparison with World Action Models. The paper overlooks the broader WAM literature, such as Fast-WAM, which explores video modeling for policy representation learning. Although the architectures differ, it is still worthwhile to have a discussion of the tradeoffs between learning a separate simulator and jointly learning world representations and actions. As a side note, I believe the latest Cosmos 3 also pivot towards such an architecture.

[1] Seeing Realism from Simulation: Efficient Video Transfer for Vision-Language-Action Data Augmentation, ICML 2026

[2] End-to-End Training for Autoregressive Video Diffusion via Self-Resampling, ECCV 2026

[3] Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion, NeurIPS 2025 spotlight

Questions for authors

See the "Critical Weaknesses" section.

Reviewer_pg7tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Sword improves an action-conditioned Wan2.2 world model for VLA policy post-training through two components: structure-guided style augmentation with depth, segmentation, and task-conditioned Cosmos-Transfer, and Dynamic Latent Bootstrapping (DLB), which reuses cached predicted latents as training context. Evaluation uses LIBERO-Spatial and LIBERO-Mixed, a raw/style-transferred mixture with disjoint training and evaluation style prompts. The paper reports better video prediction metrics and higher OpenVLA-OFT success rates after GRPO training than with WoVR.

Critical strengths
  1. The paper addresses two consequential problems for learned VLA simulators: appearance shifts and accumulated prediction errors. Table 1 includes a useful 2-by-2 component ablation rather than relying only on comparisons with different architectures. For example, SGSA alone reduces LIBERO-Mixed FVD from 213.33 to 110.71, and adding DLB further reduces it to 86.84. DLB also improves raw-data FVD from 61.57 to 34.64 without SGSA. These results support complementary empirical benefits.

  2. Reusing detached latent predictions across iterations is a practical way to introduce prediction-dependent context while retaining fixed-window training. The eight cache-design variants in Table 3 and the supplied reference code make this training idea more concrete. Holding out style prompts also provides a meaningful test beyond memorizing the training prompts, within the stated augmentation setting.

Critical weaknesses
  1. The central claim of improved utility as an RL simulator is not yet adequately established by the reported evaluation protocol. Section 4.5 and Table 2 give success rates of 61.72 percent versus 53.12 percent at step 172, but do not specify whether these policies are evaluated in the original LIBERO simulator or in their respective learned environments. The reward/success model, evaluation trial count, number of seeds, and variability are also unspecified. These distinctions are essential: improved imagined success can reflect simulator or reward-model errors rather than improved task execution. Figure 4 provides a useful action-following example, but perceptual metrics and qualitative examples cannot independently establish the physical fidelity needed for policy optimization. I am identifying an unresolved protocol issue, not asserting that evaluation was performed inside the learned model.

  2. The description and validation of DLB leave its main mechanism insufficiently resolved. Algorithm 1 hides prediction generation inside ModelStep. In the supplied reference implementation, wan_video_ours.py::training_loss writes predicted_clean_latents = latents - sigma * noise_pred to the cache after a single training forward pass, where the target latents were obtained by noising ground-truth future latents. These are denoising estimates, rather than outputs of the full inference sampling procedure. This can still be useful, and Table 1 supports an empirical benefit, but the distinction matters for the claim that cached contexts approximate the errors encountered during autoregressive inference. The paper does not characterize this discrepancy or quantify error growth by rollout horizon; Figure 5 is qualitative. A precise account of the cached estimator and evidence connecting it to long-horizon robustness are needed. The claimed cost advantage over self-rollout training also lacks measured time/memory comparisons.

Questions for authors
  1. Where exactly are the Table 2 success rates measured? Please specify the reward/success criterion and its training data, evaluation tasks and trial counts, seed count and uncertainty, and the initial policy's success rate. If available, evaluation of both resulting policies in the original LIBERO simulator would directly resolve the main concern.

  2. Does the submitted single-forward-pass clean-latent estimator correspond to the implementation used for the reported DLB results? Please define ModelStep explicitly, including noise-level sampling, and clarify how these cached estimates differ from inference-generated contexts. Do horizon-dependent errors and matched-budget sampling/time/memory measurements support the proposed explanation?

  3. For SGSA, what are the style-prompt pools and augmentation proportions, and what evidence verifies preservation of object geometry and contact events? A matched-data comparison with standard Cosmos-Transfer controls would help isolate the benefit of the proposed task/structure guidance from style augmentation itself.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.