- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on two important challenges in action-conditioned world models: (1) satisfying three key objectives, i.e., Physical Plausibility, Action Adherence, and Visual Fidelity (PAV), and (2) enhancing the model’s generalization to out-of-distribution (OOD) actions.
To tackle the first challenge, the authors introduce a reinforcement learning post-training framework based on DiffusionNFT, guiding the optimization with specifically tailored reward signals corresponding to the PAV criteria. Specifically, the authors evaluate Physical Plausibility of rollouts using VJEPA-2, leveraging its superior physical world understanding. Action Adherence is quantified by extracting embodiment-focused frames from the rollouts and feeding them into VJEPA-2 to measure the alignment between the robot's motion and the conditioning actions. Visual Fidelity is assessed via a composite of frame-level and temporal-level no-reference quality metrics.
To address the second challenge, the authors synthesize OOD actions by applying three distinct perturbations, i.e., temporal shift, structured magnitude bias, and smooth temporal noise, to non-static ground-truth actions. Then, an Out-of-Distribution Reward Function is introduced during the post-training stage to explicitly improve robustness and generalization across unseen actions.
Extensive experiments are conducted on Wan2.2-5B across multiple datasets. The results demonstrate that the proposed method outperforms existing action-conditioned world models. In addition, ablation studies validate the effectiveness of each individual design component, and downstream experiments further showcase the model’s superiority when serving as a policy evaluator.
Critical strengths
This paper makes a notable contribution by identifying and formalizing three key objectives, i.e., Physical Plausibility, Action Adherence, and Visual Fidelity (PAV), which are essential for action-conditioned world models in embodied AI. To optimize for these criteria, the authors introduce a post-training framework with carefully tailored reward signals. In particular, by leveraging the rich physical priors encoded in the latent space of VJEPA-2, the paper propose the VJEPA Reward alongside an Embodiment-focused VJEPA Reward. Comprehensive evaluations demonstrate the efficacy of this approach, offering valuable insights that could inspire subsequent work in this domain.
Critical weaknesses
-
The conceptual motivation behind enhancing OOD action generalization via post-training remains unconvincing. Post-training is typically effective at aligning and reinforcing capabilities that the base model has already acquired during pre-training, but rarely imparts entirely unseen, out-of-support capabilities. If the pre-trained base model was never exposed to these OOD actions, it is inherently difficult for post-training alone to fundamentally improve OOD generalization. Moreover, the experiments does not support the authors' claims. The reported experiments appear to be conducted on ID actions. The paper lacks dedicated evaluations or benchmarks specifically under unseen OOD actions.
-
The framework relies on a combination of multiple reward components. As evidenced by Tables 5 and 6, the final performance appears heavily dependent on delicate hyperparameter tuning. Could the authors provide a principled heuristic for determining these reward hyperparameters? Furthermore, in multi-objective reinforcement learning, conflicting gradient signals among different rewards are a common failure mode. Have the authors observed any interference between the reward objectives? To alleviate this concern, it would be highly informative if the authors could provide training curves illustrating how each individual reward evolves across training steps.
Questions for authors
-
The authors need to further clarify the meaning of Figure 4, as it is currently difficult to understand. For instance, the text claims that the proposed method mitigates the overestimation of success rate. However, it is unclear how this conclusion is directly supported by the figure.
-
For action coherence, the authors should provide a more thorough discussion and comparison between VJEPA-2 and IDMs. It remains unclear why VJEPA-2 is preferred over a standard IDM for action adherence, and how much benefit it brings.
3 comments
Thanks for the authors' response. My concerns have been resolved, and I will keep my score. Please make sure to update Figure 4 by marking the signed progression errors in the figure, as this will better substantiate the paper's claims.
Figure 4 improvement.
Thank you for suggesting improvements to Figure 4’s clarity. We will update the PDF after consolidating feedback from all reviewers.
We thank the reviewer for recognizing the PAV formulation’s value and raising questions on post-training limits, reward interactions, and policy evaluation.
W1. Why could post-training improve OOD behavior? The objection assumes that improving OOD action responses requires entirely new physical capabilities. Our setting is narrower: expert demonstrations cover a narrow action distribution, while plausible non-expert deviations can involve familiar motions and interactions. Perturbed actions are explicitly presented to the generator during post-training. They are OOD relative to expert demonstrations but are RL training inputs. Their recorded future videos are missing; our action-conditioned latent predictor supplies rewards for these samples.
10% of samples use this branch; the remainder retain gt supervision. This adapts responses beyond demonstrated actions while retaining expert-trajectory supervision. Tables 1 and 2 report prediction and ablation gains; Section 4.3 tests real-world non-expert policy executions; and the larger-shift experiment tests broader training exploration in response to Reviewer zveU, W3/Q1. These support the targeted setting. Extrapolation to much more distant actions still depends on the action-conditioned predictor's reliability—a boundary of the approach, rather than a barrier to improvement beyond the expert distribution.
W2. Coefficient selection and reward interference. The coefficients follow a concrete selection procedure, with empirical checks beyond a single tuned run. Appendix A.6 screens coefficients by whether a reward ranks recorded futures above 11 alternatives, including temporal corruptions. Table 5 reports RL-based sensitivity to VJEPA weighting and OOD sampling; AgiBot and DROID share the selected configuration. Table 2 shows that removing static regularization, embodiment-focused scoring, or visual-fidelity scoring reduces performance. Thus, the paper provides a practical heuristic and evidence for the components' complementary contributions.
Static regularization counters latent prediction scoring's preference for static outputs (Section 3.2); removing it raises predictor loss from 0.373 to 0.409 (Table 2). The ablations establish the combined objective's benefit at the tested configuration; they do not diagnose gradient alignment throughout training. Figure 7 reports PSNR; individual reward trajectories are currently unavailable. These curves will complement existing sensitivity and ablation studies with training-dynamics analysis and be included in the revised PDF.
Q1. How should Figure 4 be interpreted? The horizontal axis is average real-world task progression; the vertical axis is progression measured in world-model rollouts. Section 4.3 defines progression by completion of ordered task stages, not binary task success. Each point represents a policy-task combination; colors distinguish tasks and marker shapes distinguish the two policies. The dashed diagonal indicates perfect agreement; points above it indicate progression overestimation.
Figure 4 supports improved correspondence with real-world policy performance: correlation increases from 0.863 for Ctrl-World to 0.969 for our model. Overestimation is distinct, measured by positive deviations above the diagonal rather than correlation. We will clarify this distinction, consistently use task progression, and restrict the figure's quantitative conclusion to the reported association. Comparing absolute and signed progression errors would substantiate the calibration claim. We will also clarify rollout counts and define MMRV in the caption.
Q2. Why VJEPA-2 rather than an IDM, and what is the measured benefit? Our choice is supported by the reward interface and diagnostic below. The action-conditioned predictor forecasts latent futures from the initial observation and conditioning actions, providing a reference for scoring generated futures without recorded future videos. Embodiment-focused frames emphasize robot motion, while full frames retain scene interactions. This provides a common latent-prediction framework for physical and action-focused reward streams. The relevant distinction is the scoring design and measured performance; an IDM's inverse direction is not itself a disadvantage.
We also compared evaluators on DROID using the same 12-candidate GT-selection diagnostic:
AC-VJEPA achieves 88% GT-selection accuracy, six percentage points above the strongest tested IDM. This quantitatively supports the evaluator choice alongside its compatibility with action-conditioned latent prediction. The diagnostic tests discrimination among recorded, generated, and corrupted futures, including temporal action mismatches. Its scope is evaluator selection under this protocol; isolating downstream RL gains from an IDM-to-VJEPA reward substitution requires a separate controlled training comparison. We will provide evaluator settings and diagnostic sample counts.