- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a novel OOD-aware World Model (WM) training paradigm which aims to reduce long-horizon autoregressive rollout degradation under potentially OOD action-scene combinations. First, the authors empirically motivate the key observation that, beyond well-studied autoregressive model error accumulation, actions which were unforeseen in the training data can drastically destabilize a rollout. To mitigate this effect, this work proposes a WM training paradigm which consists of a state-action verifier to assess the support of a proposed action under the visual context, a novel gating mechanism for conditioning on the action dependent on the validity, and two-stage training pipeline which explicitly trains on unsupported actions to stabilize the model. Experimental results in simulation demonstrate improved model long horizon predictions under OOD actions, while maintaining competitive performance in purely ID settings.
Critical strengths
This paper seeks to address an important problem in consistency and reliability long-horizon autoregressive action-conditioned world models. The specific challenge in this work (stability under “unsupported” actions) is intuitive and well-motivated empirically. The proposed pipeline is reasonable and the empirical results are compelling, with clear gains in prediction error in OOD settings, while maintaining reasonable performance in purely ID settings.
Critical weaknesses
The proposed method is dependent on an anticipated family of OOD actions, used during the fine-tuning process, which may massively inhibit the generality of the proposed method. It is not clear how this method would perform when evaluated on a family of actions which differs substantially from the anticipated actions. Similarly, all elements of the training pipeline integrate some learned component (e.g., the verifier classifier, or MLPs in the gating mechanism), which reinforces concerns about the generalizability of a model trained with the proposed framework. While this is discussed as potential future work, expanding the evaluation scope OOD evaluation setting (with unforeseen actions) would strengthen the demonstrated utility of this framework.
Questions for authors
How is the OOD action family determined? How do the authors anticipate the model would perform with an unforeseen action/scene pair?
What is the reason behind the method underperforming the nominal model in ID scenarios, and how can this gap be reconciled?
While the proposed method shows improvement over baselines in OOD settings, there is still substantial degradation (e.g., LPIPS ~= 0.46). What is an acceptable margin of error for, e.g., planning for robot manipulation?
3 comments
Question 2: Underperformance under ID conditions and possible remedies
Response. We appreciate the opportunity to clarify the differences under ID actions.
First, DG-ZA does not uniformly underperform Original under ID conditions. At the 16-second horizon, the results are:
| Dataset | Original FID ↓ | DG-ZA FID ↓ |
|---|---|---|
| Habitat-Sim | 86.551 | 37.344 |
| WorldGym | 96.017 | 90.912 |
In particular, DG-ZA substantially improves upon Original on Habitat-Sim under ID actions.
However, DG-ZA is less competitive than certain ID-specialized gating baselines. For example, Fixed Decline achieves an ID FID of 20.304 on Habitat-Sim at 16 seconds, compared with 37.344 for DG-ZA.
A plausible explanation is the difference in training objectives. As detailed in Appendix C.3, Original and the ID-only gating baselines continue optimizing their standard ID objectives. In contrast, DG-ZA undergoes OOD-aware post-tuning, jointly optimizing the standard ID diffusion objective and an additional OOD consistency objective. The ID-only baselines can therefore specialize entirely in supported-action generation, whereas DG-ZA must also accommodate robustness to unsupported actions.
Potential directions for improving the balance between ID and OOD performance include refining the weighting or scheduling of the two training objectives.
Another possibility is verifier-guided model routing, analogous to a mixture-of-experts (MoE) design. Since our framework already includes a state–action verifier that estimates whether a transition is supported, its validity score could serve as a routing signal: supported state–action pairs could be handled by an ID-specialized model, while unsupported pairs could be routed to our OOD-robust model.
We view these as promising directions for further improving ID performance while retaining the robustness benefits of validity-aware action gating.
Question 3: Acceptable prediction error for robot manipulation planning
Response. We appreciate this question and agree that substantial prediction errors remain, particularly in robotic manipulation settings.
However, perceptual rollout metrics alone cannot determine whether the remaining errors are acceptable for downstream planning. Their practical significance depends on the task objectives and how prediction errors affect subsequent decisions. Establishing an acceptable error margin would therefore require task-specific evaluation rather than a fixed threshold on perceptual metrics.
Our primary objective is to characterize and mitigate long-horizon rollout degeneration under unsupported actions. As discussed in Appendix H, an important direction for future work is investigating whether the resulting improvements in rollout stability can benefit downstream planning.
For example, in model-predictive control (MPC) with the cross-entropy method (CEM), candidate action sequences may contain actions poorly supported by the learned dynamics, since they can be proposed by a separately trained policy or planning procedure.
In such settings, the state–action verifier could inform how the planner handles unreliable predictions. Depending on the task, candidate rollouts could be penalized according to their validity estimates, terminated when unreliable transitions are detected, or continued while accounting for their uncertainty.
Determining whether these strategies improve planning success, and under what conditions continuation is preferable to abstention, requires task-specific evaluation.
We consider investigating these downstream applications and their planning performance an important direction for future work.
Manuscript revisions
We have clarified the rationale for the predefined OOD action construction and its generalization scope in Appendix H, and added the action extrapolation experiments to Section 5.3 and Appendix D.5. We have also expanded the discussion of potential downstream planning applications in Appendix H.
Gentle Follow-up on Author Response
Thank you again for your careful assessment and helpful feedback.
As the discussion period draws to a close, we would like to briefly follow up on our rebuttal. We have provided additional action extrapolation results and clarified the limitations of our OOD construction, the performance under ID actions, and the potential implications for downstream planning.
We hope these responses have helped address your main concerns and would be grateful to hear whether any questions remain or further clarification would be helpful. If our responses have addressed your main concerns, we would also appreciate your consideration of an updated assessment.
Thank you again for your time and consideration.
We are grateful for the reviewer's thoughtful engagement with our work. We appreciate the concerns regarding generalization to unforeseen actions, performance under supported actions, and the practical significance of the remaining prediction errors. These questions help clarify the scope and limitations of our contribution.
Weakness 1 / Question 1: Dependence on anticipated OOD actions and generalization to unforeseen state–action pairs
Response. We agree that dependence on an anticipated OOD action family is an important consideration.
As discussed in Appendix H (Scope of Unsupported Actions), constructing an exhaustive dataset of unsupported state–action pairs would require determining whether every candidate pair lies outside the support of the entire training distribution. Such state-dependent support estimation is impractical at scale.
We therefore adopt a practical, controlled construction: sampling actions outside the marginal training-action range. This provides unambiguous unsupported-action conditions and allows us to systematically investigate their effects on autoregressive generation.
We acknowledge that this construction does not directly evaluate every possible state-conditional incompatibility, including cases in which an action is marginally plausible but unsupported under the current visual state. Such mismatches are nevertheless included as negative examples when training the state–action verifier, as described in Appendix C.2.
To investigate robustness beyond the anticipated OOD action range, we conducted additional Near and Far action extrapolation experiments, now included in Section 5.3 and Appendix D.5.
Both conditions sample actions outside the OOD-All post-tuning range along all three action dimensions, with Far extending farther beyond the range than Near. We evaluate Original and DG-ZA under these settings without further training.
The original OOD-All post-tuning ranges and extrapolation ranges are:
The 16-second Habitat-Sim results are:
DG-ZA improves all four metrics under both extrapolation settings, reducing 16-second FID by 32.7% under Near and 18.2% under Far relative to Original.
We additionally aggregated the state–action verifier's outputs over the complete 16-second autoregressive rollouts. 99.11% and 99.17% of evaluated state–action pairs under Near and Far, respectively, were classified as OOD (q<0.5). For reference, the Habitat-Sim state–action verifier achieves 89.70% accuracy and 0.898 AUROC on its original verifier test distribution, as reported in Appendix C.2. The original verifier test set includes challenging state–action mismatches constructed by pairing states with actions from different time steps or trajectories, which may partly account for its imperfect classification accuracy.
These findings show that the verifier continues assigning low validity to extrapolated actions and that DG-ZA maintains improvements in long-horizon rollout stability beyond the action range used during OOD-aware post-tuning.