acceptodds
Under review as a conference paper at ICLR 2027

Rollout Degeneration under Unsupported Actions in World Models

Abstract

Autoregressive world models often degenerate over long rollouts, a problem typically attributed to the train-test mismatch between clean training contexts and imperfect model-generated visual contexts. We show that world models face an additional source of instability: externally supplied actions can induce state-action pairs outside the regime covered during training. Through controlled action-space manipulations, we find that such *unsupported actions* substantially destabilize ongoing rollouts. Beyond the conventional degeneration pattern of progressive off-support drift, this instability can manifest as *conservative collapse*, where severely degraded visual states abruptly transition to visually plausible but dynamically unreliable states. Based on this analysis, we propose *validity-aware action gating* to mitigate degeneration under unsupported actions by estimating whether the state-action pair lies within the training support and adapting the influence of action conditioning accordingly. Experiments in visual navigation and robotic manipulation show that our method consistently reduces late-horizon degeneration under unsupported actions, lowering final 16-second FID by 45.3% and 13.3% over the original models in the two domains, respectively, while preserving competitive performance under supported actions.

Primary area:
generative models
Keywords:
world models, long-horizon degeneration, exposure bias, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UsjDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel OOD-aware World Model (WM) training paradigm which aims to reduce long-horizon autoregressive rollout degradation under potentially OOD action-scene combinations. First, the authors empirically motivate the key observation that, beyond well-studied autoregressive model error accumulation, actions which were unforeseen in the training data can drastically destabilize a rollout. To mitigate this effect, this work proposes a WM training paradigm which consists of a state-action verifier to assess the support of a proposed action under the visual context, a novel gating mechanism for conditioning on the action dependent on the validity, and two-stage training pipeline which explicitly trains on unsupported actions to stabilize the model. Experimental results in simulation demonstrate improved model long horizon predictions under OOD actions, while maintaining competitive performance in purely ID settings.

Critical strengths

This paper seeks to address an important problem in consistency and reliability long-horizon autoregressive action-conditioned world models. The specific challenge in this work (stability under “unsupported” actions) is intuitive and well-motivated empirically. The proposed pipeline is reasonable and the empirical results are compelling, with clear gains in prediction error in OOD settings, while maintaining reasonable performance in purely ID settings.

Critical weaknesses

The proposed method is dependent on an anticipated family of OOD actions, used during the fine-tuning process, which may massively inhibit the generality of the proposed method. It is not clear how this method would perform when evaluated on a family of actions which differs substantially from the anticipated actions. Similarly, all elements of the training pipeline integrate some learned component (e.g., the verifier classifier, or MLPs in the gating mechanism), which reinforces concerns about the generalizability of a model trained with the proposed framework. While this is discussed as potential future work, expanding the evaluation scope OOD evaluation setting (with unforeseen actions) would strengthen the demonstrated utility of this framework.

Questions for authors

How is the OOD action family determined? How do the authors anticipate the model would perform with an unforeseen action/scene pair?

What is the reason behind the method underperforming the nominal model in ID scenarios, and how can this gap be reconciled?

While the proposed method shows improvement over baselines in OOD settings, there is still substantial degradation (e.g., LPIPS ~= 0.46). What is an acceptable margin of error for, e.g., planning for robot manipulation?

3 comments

We are grateful for the reviewer's thoughtful engagement with our work. We appreciate the concerns regarding generalization to unforeseen actions, performance under supported actions, and the practical significance of the remaining prediction errors. These questions help clarify the scope and limitations of our contribution.

Weakness 1 / Question 1: Dependence on anticipated OOD actions and generalization to unforeseen state–action pairs

Response. We agree that dependence on an anticipated OOD action family is an important consideration.

As discussed in Appendix H (Scope of Unsupported Actions), constructing an exhaustive dataset of unsupported state–action pairs would require determining whether every candidate pair lies outside the support of the entire training distribution. Such state-dependent support estimation is impractical at scale.

We therefore adopt a practical, controlled construction: sampling actions outside the marginal training-action range. This provides unambiguous unsupported-action conditions and allows us to systematically investigate their effects on autoregressive generation.

We acknowledge that this construction does not directly evaluate every possible state-conditional incompatibility, including cases in which an action is marginally plausible but unsupported under the current visual state. Such mismatches are nevertheless included as negative examples when training the state–action verifier, as described in Appendix C.2.

To investigate robustness beyond the anticipated OOD action range, we conducted additional Near and Far action extrapolation experiments, now included in Section 5.3 and Appendix D.5.

Both conditions sample actions outside the OOD-All post-tuning range along all three action dimensions, with Far extending farther beyond the range than Near. We evaluate Original and DG-ZA under these settings without further training.

The original OOD-All post-tuning ranges and extrapolation ranges are:

ConditionΔx\Delta x (m)∣Δy∣\lvert\Delta y\rvert (m)∣Δψ∣\lvert\Delta\psi\rvert (rad)
OOD-All post-tuning[0.35, 0.50][0.15, 0.30][1.00, 1.50]
Near[0.55, 0.65][0.35, 0.40][1.55, 1.80]
Far[0.70, 0.80][0.45, 0.55][1.85, 2.10]

The 16-second Habitat-Sim results are:

ConditionModelLPIPS ↓DreamSim ↓FID ↓Support ↓
NearOriginal0.4740.36074.95877.365
DG-ZA0.4580.30150.47373.209
FarOriginal0.4690.33468.09876.210
DG-ZA0.4620.30255.70073.207

DG-ZA improves all four metrics under both extrapolation settings, reducing 16-second FID by 32.7% under Near and 18.2% under Far relative to Original.

We additionally aggregated the state–action verifier's outputs over the complete 16-second autoregressive rollouts. 99.11% and 99.17% of evaluated state–action pairs under Near and Far, respectively, were classified as OOD (q<0.5q < 0.5). For reference, the Habitat-Sim state–action verifier achieves 89.70% accuracy and 0.898 AUROC on its original verifier test distribution, as reported in Appendix C.2. The original verifier test set includes challenging state–action mismatches constructed by pairing states with actions from different time steps or trajectories, which may partly account for its imperfect classification accuracy.

These findings show that the verifier continues assigning low validity to extrapolated actions and that DG-ZA maintains improvements in long-horizon rollout stability beyond the action range used during OOD-aware post-tuning.

Question 2: Underperformance under ID conditions and possible remedies

Response. We appreciate the opportunity to clarify the differences under ID actions.

First, DG-ZA does not uniformly underperform Original under ID conditions. At the 16-second horizon, the results are:

DatasetOriginal FID ↓DG-ZA FID ↓
Habitat-Sim86.55137.344
WorldGym96.01790.912

In particular, DG-ZA substantially improves upon Original on Habitat-Sim under ID actions.

However, DG-ZA is less competitive than certain ID-specialized gating baselines. For example, Fixed Decline achieves an ID FID of 20.304 on Habitat-Sim at 16 seconds, compared with 37.344 for DG-ZA.

A plausible explanation is the difference in training objectives. As detailed in Appendix C.3, Original and the ID-only gating baselines continue optimizing their standard ID objectives. In contrast, DG-ZA undergoes OOD-aware post-tuning, jointly optimizing the standard ID diffusion objective and an additional OOD consistency objective. The ID-only baselines can therefore specialize entirely in supported-action generation, whereas DG-ZA must also accommodate robustness to unsupported actions.

Potential directions for improving the balance between ID and OOD performance include refining the weighting or scheduling of the two training objectives.

Another possibility is verifier-guided model routing, analogous to a mixture-of-experts (MoE) design. Since our framework already includes a state–action verifier that estimates whether a transition is supported, its validity score could serve as a routing signal: supported state–action pairs could be handled by an ID-specialized model, while unsupported pairs could be routed to our OOD-robust model.

We view these as promising directions for further improving ID performance while retaining the robustness benefits of validity-aware action gating.

Question 3: Acceptable prediction error for robot manipulation planning

Response. We appreciate this question and agree that substantial prediction errors remain, particularly in robotic manipulation settings.

However, perceptual rollout metrics alone cannot determine whether the remaining errors are acceptable for downstream planning. Their practical significance depends on the task objectives and how prediction errors affect subsequent decisions. Establishing an acceptable error margin would therefore require task-specific evaluation rather than a fixed threshold on perceptual metrics.

Our primary objective is to characterize and mitigate long-horizon rollout degeneration under unsupported actions. As discussed in Appendix H, an important direction for future work is investigating whether the resulting improvements in rollout stability can benefit downstream planning.

For example, in model-predictive control (MPC) with the cross-entropy method (CEM), candidate action sequences may contain actions poorly supported by the learned dynamics, since they can be proposed by a separately trained policy or planning procedure.

In such settings, the state–action verifier could inform how the planner handles unreliable predictions. Depending on the task, candidate rollouts could be penalized according to their validity estimates, terminated when unreliable transitions are detected, or continued while accounting for their uncertainty.

Determining whether these strategies improve planning success, and under what conditions continuation is preferable to abstention, requires task-specific evaluation.

We consider investigating these downstream applications and their planning performance an important direction for future work.

Manuscript revisions

We have clarified the rationale for the predefined OOD action construction and its generalization scope in Appendix H, and added the action extrapolation experiments to Section 5.3 and Appendix D.5. We have also expanded the discussion of potential downstream planning applications in Appendix H.

AuthorsOpenReview ↗

Gentle Follow-up on Author Response

Thank you again for your careful assessment and helpful feedback.

As the discussion period draws to a close, we would like to briefly follow up on our rebuttal. We have provided additional action extrapolation results and clarified the limitations of our OOD construction, the performance under ID actions, and the potential implications for downstream planning.

We hope these responses have helped address your main concerns and would be grateful to hear whether any questions remain or further clarification would be helpful. If our responses have addressed your main concerns, we would also appreciate your consideration of an updated assessment.

Thank you again for your time and consideration.

Reviewer_DRrPOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates a form of rollout degeneration in action-conditioned world models caused by unsupported actions, i.e., state–action pairs that lie outside the training support. The authors identify two failure modes:

  1. Off-support drift, in which rollouts degrade progressively over the horizon.
  2. Conservative collapse, in which rollouts degrade and then abruptly return to visually plausible but dynamically unreliable frames.

Based on this analysis, the authors propose validity-aware action gating, which estimates whether a state–action pair is supported and attenuates the action conditioning accordingly. Experiments on NWM (Habitat-Sim) and WorldGym (BridgeData V2) show that the method reduces FID under unsupported actions.

Critical strengths
  1. The paper is very well written.
    • The figures are clear and compelling. (esp. Figs, 1, 3, 11, 12--- Fig 4 is nice too but less surprising)
    • The narrative is extremely compelling and easy to follow.
  2. The idea that OOD actions destabilize autoregressive models is not new (see Weakness 1), so I was surprised by how much I enjoyed the paper. In particular, it contains a few salient findings that I found valuable precisely because they are somewhat pathological:
    • One of the most interesting points is in lines 291–293, where the paper argues that visual-continuity applications may still benefit from continuing a rollout after an unsupported action, provided its destabilizing effect is attenuated (see Q2). This point deserves more elaboration; it was a shame that more was not said about it.
    • The notion of conservative collapse itself. My prior would have been to categorize a rollout that degrades and then abruptly returns to plausible frames as a strange aberration rather than a systematic phenomenon. The paper shows that it recurs across settings (RECON and all three structured OOD conditions in Habitat-Sim) and argues that the plausible frames should not be taken as evidence that the model has recovered the correct dynamics.
Critical weaknesses

I have few complaints about the paper; almost every concern I had was pre-empted. For example, I was initially worried that Gaussian-perturbed actions would not be representative of OOD actions, but the Mahalanobis analysis in Appendix A.1 addresses this. The two weaknesses I would most like to see addressed are:

  1. Some important literature is missing. World models are the purview of both RL and vision; this paper is solidly grounded in the vision tradition but omits much of the relevant work on offline model-based RL (e.g., MOPO, MOReL), which strongly informs world models. These methods use model uncertainty to detect unsupported state–action pairs and penalize or truncate rollouts that reach them, a direct precedent for both the verifier and the option of abstaining from further simulation (lines 290–291). This matters for positioning, since the paper currently frames its novelty only against state-centric vision work.
    • In particular, the authors should engage with the formal analysis of compounding errors in seminal imitation-learning work such as DAgger (Ross et al., 2011); the paper cites scheduled sampling, but not this line of work.
  2. No confidence intervals are reported in either the plots or the tables. This matters most for Tables 1 and 2, where several differences are small and each variant appears to come from a single training run. For example, the evidence for dual gating in Table 1 is SG-ZA vs. DG-ZA at 16 seconds (LPIPS 0.465 vs. 0.454, FID 31.7 vs. 27.0). Bootstrap intervals over the 1,000 Habitat-Sim test trajectories would be inexpensive to add. Relatedly, I think more emphasis on the experimental analysis would be good: the discussion of Tables 1 and 2 reports which variant wins but not why. The implementation details in lines 419–427, which are repeated in Appendix C.3, could be moved to the appendix to make room.

Minor organizational suggestions:

  • Figure 1 illustrates both patterns, but its off-support drift example uses ID actions; an OOD-induced drift example near Figure 3 would make Section 3.2 easier to follow.
  • Include qualitative examples of the robotic manipulation experiments in the main body, although I recognize that space is limited.
  • Provide slightly more context on RAE-NWM.
Questions for authors
  1. Why did you evaluate both single-gate and dual-gate variants? It was not clear to me why this comparison is important to the narrative arc of the paper.
  2. Could you expand on the claim that visual-continuity applications may still benefit from continuing the rollout after an unsupported action? I understand that this is the purpose of the gating method, but more intuitive prose would be helpful.
  3. Appendix F.1 notes that, within a single OOD condition, some rollouts drift progressively, some remain severely degraded, and some undergo one or more conservative collapses. What determines which pattern a given rollout follows? Could you report how often each pattern occurs per condition (e.g., per σ\sigma on RECON)? Interestingly, the RECON curves suggest that milder perturbations (σ=0.5\sigma = 0.5) are the ones that cross the ID curve, while σ=10\sigma = 10 degrades nearly monotonically (Table 6). Is there a perturbation magnitude beyond which rollouts never return to visually plausible frames?
6 comments

We sincerely thank the reviewer for the encouraging evaluation and thoughtful feedback. We are especially grateful that you found the paper engaging, and we appreciate your interest in conservative collapse and its broader implications. Your detailed suggestions have been particularly helpful in strengthening both the experimental analysis and the positioning of our work.

Weakness 1: Missing connections to model-based RL and imitation learning

Response. We thank the reviewer for highlighting these important connections to offline model-based RL and imitation learning. We have revised the Introduction and Related Work to incorporate MOPO (Yu et al., 2020), MOReL (Kidambi et al., 2020), and DAgger (Ross et al., 2011) and clarify our positioning.

We agree that state–action support mismatch and uncertainty-aware model usage have been studied extensively in offline RL. MOPO penalizes rewards based on predictive uncertainty from a probabilistic dynamics ensemble, while MOReL employs an unknown state–action detector and pessimistic HALT transitions. We acknowledge these as precedents for uncertainty-aware safeguards and simulation abstention.

Our work builds on this established perspective but focuses on a different problem: characterizing how unsupported actions affect long-horizon degeneration in autoregressive generative visual world models. Rather than using dynamics uncertainty to safeguard policy learning, DG-ZA employs a separately trained state–action support verifier to modulate action conditioning directly during generation.

We also acknowledge DAgger's formal analysis of compounding errors under distribution shift. As discussed in the revised Related Work, behavioral cloning can incur worst-case excess costs growing quadratically with the task horizon, whereas DAgger mitigates learner-induced distribution shift through iterative expert labeling and dataset aggregation.

While this analysis concerns policy execution, our work empirically investigates an additional source of rollout instability arising from externally supplied unsupported actions, beyond errors accumulated in model-generated visual contexts.

Weakness 2: Missing confidence intervals and insufficient interpretation of Tables 1 and 2

Response. Following the reviewer's suggestion, we conducted 1,000 paired bootstrap replicates for both Habitat-Sim and WorldGym and incorporated the results into Appendices D.4 and E.5.

For Habitat-Sim (Table 1), we use a paired cluster bootstrap over 1,000 original source trajectories, each contributing eight evaluation windows, for a total of 8,000 evaluation windows. All windows from a sampled source trajectory are retained together, and the same sampled source trajectories are reused across models.

At 16 seconds, the 95% confidence intervals for DG-ZA minus each comparator are:

ComparatorLPIPSDreamSimFID
Original[-0.02119, -0.01739][-0.05861, -0.05130][-23.931, -20.851]
SG-A[-0.01700, -0.01360][-0.05383, -0.04708][-26.923, -24.006]
DG-A[-0.01409, -0.01064][-0.04575, -0.03934][-21.026, -18.443]
SG-ZA[-0.01259, -0.00939][-0.02271, -0.01646][-5.866, -3.518]
DG-ZA w/o post[-0.01445, -0.01139][-0.03523, -0.02878][-16.786, -14.148]

All confidence intervals exclude zero, providing statistical support for the improvements of DG-ZA over the evaluated comparison models.

For Table 2 (Table 3 in the revised manuscript), we use a paired trajectory-level bootstrap over WorldGym test trajectories with 1,000 replicates to estimate 95% confidence intervals for DG-ZA minus Original.

The resulting FID confidence intervals are:

HorizonIDOOD
4s[-3.977, 4.044][-13.038, -2.104]
8s[-4.981, 4.945][-11.777, -2.810]
12s[-7.058, 2.075][-18.564, -5.478]
16s[-6.624, 5.639][-19.604, -1.351]

The OOD intervals consistently exclude zero, whereas the ID intervals include zero. These results provide additional statistical support for the OOD improvements across the evaluated horizons, without a statistically significant degradation in ID performance under the current evaluation.

Ablation interpretation. We have also expanded the discussion in Section 5.2 to explain the contributions of the individual design choices rather than merely identifying the best-performing variants.

First, incorporating state–action validity produces particularly substantial improvements. These improvements are consistent with our central hypothesis that action support should be evaluated jointly with the current visual state rather than solely through the marginal action distribution.

Second, explicitly separating ID and OOD gating schedules further improves performance under both verifier variants, and the contribution of this architectural choice is discussed below.

Question 1: Why compare single-gate and dual-gate variants?

Response. The comparison tests whether explicitly separating supported and unsupported denoising behaviors provides a useful structural inductive bias.

The single-gate variant predicts action-conditioning strength using a unified function of the denoising timestep and validity score. It must therefore learn how the conditioning schedule should differ between supported and unsupported transitions within one parameterization.

By contrast, the dual-gate variant maintains separate learned timestep-dependent schedules for ID and OOD transitions, which are softly interpolated according to the estimated validity score.

This design incorporates our hypothesis—that supported and unsupported transitions may benefit from different denoising-time conditioning behaviors—directly into the architecture, rather than requiring a single gating function to implicitly distinguish the two regimes. We have explicitly clarified this motivation in Section 4.2.

The ablation in Table 1 supports this motivation. DG-ZA reduces 16-second OOD-All FID from 31.71 to 27.02 relative to SG-ZA, alongside improvements in LPIPS, DreamSim, and support distance.

We therefore interpret the comparison as evidence that explicitly separating the ID and OOD schedules provides a useful inductive bias for learning validity-aware action conditioning.

Question 2: Why continue a rollout after encountering an unsupported action?

Response. We appreciate the reviewer's interest in this point. We have expanded the motivation in Section 4 and the discussion of downstream implications in Appendix H.

We distinguish two application settings in which the appropriate response to an unsupported action may differ.

Real-time interactive simulation. Consider a real-time interactive simulator where a user repeatedly issues movement commands toward unreachable locations, such as positions inside a wall, beneath the floor, or high in the sky, encountering situations rarely represented in the training data.

While the resulting dynamics may be uncertain, attenuating the influence of such unsupported commands may prevent local prediction errors from propagating into persistent visual degeneration, allowing the interactive session to remain visually coherent.

Model-based planning with expensive candidate rollouts. Another possible application is Model Predictive Control (MPC) using the Cross-Entropy Method (CEM), as discussed in Appendix H.

In MPC/CEM, candidate action sequences are evaluated through rollouts generated by a learned dynamics model. Since action proposals may come from a separately trained policy or a planning procedure, they are not necessarily restricted to the state–action distribution on which the dynamics model was trained. Consequently, candidate rollouts may contain actions poorly supported by the learned dynamics.

When an unsupported action is detected early in a candidate rollout, immediately rejecting the candidate may be computationally efficient. However, when such an action occurs near the end of a long and expensive rollout, rejecting the candidate also discards the computation invested in the preceding trajectory.

One possible alternative is validity-aware continuation, in which the planner continues generating a candidate rollout while accounting for the estimated validity of its state–action transitions. The planner could incorporate validity scores into its evaluation of candidate action sequences, penalizing unreliable predictions rather than relying solely on the predicted task objective. Depending on the task, a planner could instead terminate unreliable candidates or trigger replanning.

Thus, continuation and abstention need not be mutually exclusive. The validity verifier could support different strategies according to task requirements, computational budget, and the reliability needed for decision-making.

We consider investigating these alternatives and their downstream planning performance an important direction for future work.

Reviewer_xs6QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how out-of-distribution (OOD) actions contribute to long-horizon degeneration in autoregressive world models. The authors identify two failure patterns, off-support drift and conservative collapse, and propose validity-aware action gating to mitigate these issues. The method uses a state-action verifier to adjust action-conditioning strength, together with OOD-aware post-training. Experiments on navigation and robotic manipulation world models show improved rollout metrics under OOD actions.

Critical strengths
  1. The paper provides an interesting analysis of how unsupported actions affect world model rollouts, highlighting an issue beyond conventional exposure bias.
  2. The proposed method is relatively simple and can be applied to existing world models. The experiments across navigation and manipulation domains, together with the ablation studies, provide reasonable evidence of improved rollout stability.
Critical weaknesses
  1. Limited novelty and failure to address the underlying problem. The proposed method mainly reduces the influence of OOD actions rather than improving the world model’s ability to predict their consequences. While this may improve visual stability, it does not fundamentally address the lack of generalization to unseen state-action pairs. The gating mechanism itself is also relatively incremental compared with existing approaches.
  2. Unclear practical significance of the improvements. Although the paper reports substantial FID improvements, other metrics show relatively modest gains. More importantly, these metrics primarily measure visual similarity rather than whether the generated trajectories correctly follow the commanded actions. Suppressing action conditioning could produce visually plausible but dynamically incorrect predictions, which is particularly concerning for world models used in planning and control. The qualitative comparisons in the appendix also do not sufficiently establish improvements in action fidelity.
  3. Limited downstream validation. The experiments focus on rollout quality under synthetic OOD action perturbations, without demonstrating whether the improvements benefit actual downstream tasks. It remains unclear whether the proposed method improves planning, policy learning, or decision-making, where accurately predicting action consequences is more important than maintaining visual plausibility.
Questions for authors
  1. When an action is classified as OOD, the proposed method reduces its influence on generation. How does this improve the accuracy of the predicted dynamics rather than simply producing more conservative or visually plausible outputs?
  2. Can the authors quantitatively evaluate action controllability or action-conditioned prediction accuracy? In particular, does the gating mechanism introduce a tradeoff between visual quality and following the commanded actions?
  3. Can the authors demonstrate improvements on downstream tasks such as model-based planning or policy evaluation? It would be helpful to understand whether the reported metric improvements translate into practical benefits.
  4. How well does the state-action verifier generalize to unseen OOD actions beyond the synthetic perturbations used during training? How sensitive is the overall method to incorrect validity predictions?
  5. Can the authors provide more convincing qualitative comparisons showing that the proposed method preserves correct action-dependent dynamics, rather than simply generating visually stable trajectories?
4 comments

We appreciate the reviewer's careful evaluation and constructive feedback. We would first like to emphasize the central objective of our work: we aim to identify and mitigate long-horizon rollout degeneration induced by unsupported actions, rather than accurately extrapolate the dynamics of arbitrary unseen state–action pairs. We recognize that attenuating unsupported action conditioning cannot guarantee accurate predictions of the resulting state transitions. This distinction is essential to interpreting our contributions and results.

Weakness 1 / Questions 1 and 5: Novelty, unsupported dynamics, and qualitative comparisons

Response. We agree that gating and validity estimation are not individually new techniques. However, our contribution extends beyond applying an existing gating operator.

Our conceptual and empirical novelty lies in identifying unsupported actions as an additional source of degeneration in autoregressive generative world models, characterizing the resulting failure patterns and investigating how state–action validity estimation can be integrated with denoising-time action gating to mitigate this instability.

Although our gating design draws inspiration from RAE-NWM's time-driven dynamics gating, RAE-NWM was not developed to address action-induced degeneration. Our analysis reveals that unsupported actions can abruptly destabilize ongoing rollouts and, in some cases, induce conservative collapse, where severely degraded predictions subsequently return to visually plausible but dynamically unreliable states. Motivated by these findings, our formulation combines state–action support estimation with separate denoising-time conditioning schedules for supported and unsupported transitions, yielding a practical mitigation of the identified degeneration rather than a general solution to OOD dynamics prediction.

This distinction also addresses the request for further qualitative evidence of correct action-dependent dynamics. As explained in Appendix F.2, our existing qualitative comparisons initialize all models from identical contexts and execute them under identical OOD-All action sequences. Because these actions lie outside the learned state–action support, their consequences are not reliably constrained by training, and the predicted rollouts may diverge substantially across methods.

We believe these comparisons are appropriate for illustrating rollout degeneration and the visual stability achieved by DG-ZA, which are the intended objects of our study. They are not presented as evidence that DG-ZA accurately extrapolates arbitrary unsupported dynamics. We complement these qualitative observations with the quantitative counterfactual action-response evaluation described below.

Weakness 2 / Questions 1 and 2: Visual stability versus action-conditioned prediction accuracy

Response. We agree that improved visual metrics do not necessarily imply more accurate action-conditioned dynamics. We therefore conducted an additional one-step counterfactual action-response experiment, now reported in Appendix G.2.

Directly estimating the executed action from generated frames is non-trivial, as it requires a reliable inverse-dynamics estimator, which may struggle with severely degenerated observations and whose prediction errors could confound the evaluation. Instead, we use Habitat-Sim to obtain paired ground-truth outcomes for alternative actions and evaluate whether the model's predictions respond to the commanded actions.

We construct 256 counterfactual pairs for each of the ID and OOD settings. Each pair starts from the same initial visual context and uses two actions with opposite yaw changes: Δψ=±0.60\Delta\psi = \pm 0.60 rad for ID and Δψ=±1.25\Delta\psi = \pm 1.25 rad for OOD.

The simulator produces the corresponding ground-truth observations y+y^+ and y−y^-. We then generate predictions y^+\hat{y}^+ and y^−\hat{y}^- conditioned on the respective actions, keeping the initial context and diffusion noise fixed.

We define the action-response score as

Sd=12[d(y^+,y−)+d(y^−,y+)−d(y^+,y+)−d(y^−,y−)]S_d = \frac{1}{2}\left[d(\hat{y}^+,y^-) + d(\hat{y}^-,y^+) - d(\hat{y}^+,y^+) - d(\hat{y}^-,y^-)\right]

where dd denotes LPIPS or DreamSim. A positive score indicates that the predictions are, on average, perceptually closer to the ground-truth outcomes associated with their commanded actions than to those of the alternative actions.

The results at the one-step prediction horizon are:

RegimeMetricOriginal ↑DG-ZA ↑
IDLPIPS0.16880.0783
IDDreamSim0.12560.0708
OODLPIPS0.14280.0343
OODDreamSim0.15240.0546

DG-ZA achieves positive action-response scores under both ID and OOD actions, indicating that its predictions reflect changes in the commanded actions.

While Original achieves higher scores in this one-step evaluation, this evaluation does not capture the accumulated degeneration that arises during autoregressive generation, which our long-horizon rollout experiments are designed to assess.

As shown in Table 1, DG-ZA improves 16-second LPIPS from 0.473 to 0.454, DreamSim from 0.349 to 0.294, and FID from 49.36 to 27.02 relative to Original under OOD-All actions. Unlike the one-step counterfactual experiment, these long-horizon results account for the accumulated effects of autoregressive generation, with the improvements in rollout-level LPIPS and DreamSim indicating better perceptual alignment with the corresponding simulator outcomes under the evaluated OOD action sequences.

Taken together, these results highlight the value of DG-ZA in mitigating long-horizon degeneration under unsupported actions, while also revealing differences in short-horizon action responsiveness. The importance of these properties depends on the intended application. For example, real-time interactive video generation and entertainment-oriented simulations may particularly benefit from maintaining visual continuity under unusual user inputs, whereas applications requiring precise short-horizon action-conditioned dynamics may place greater emphasis on action fidelity. DG-ZA thus provides a robustness-oriented approach that addresses a limitation of existing world models and expands the available design choices for different applications.

Weakness 3 / Question 3: Limited downstream validation

Response. We appreciate the reviewer's interest in the practical implications of our findings. Our present study focuses on characterizing and mitigating action-induced degeneration in generative world models. Establishing improvements in planning or policy-learning performance would require a separate task-dependent evaluation framework, including a planner or policy, goal specifications, and appropriate control metrics.

We therefore regard downstream planning as an important direction for future work rather than a demonstrated contribution of this paper.

As discussed in Appendix H, one possible application is validity-aware MPC/CEM planning. In this setting, candidate action sequences are evaluated through world-model rollouts, and unsupported actions may appear within those sequences because the action proposals are not necessarily restricted to the dynamics model's training distribution.

The verifier could help identify unreliable transitions, allowing a planner to penalize low-confidence predictions, terminate unreliable candidate rollouts, or trigger replanning.

Investigating whether such strategies translate improved rollout stability into better downstream decision-making requires task-specific evaluation, which we leave for future work.

Question 4: Generalization to unseen actions and sensitivity to verifier errors

Response. We conducted additional action extrapolation experiments, now included in Section 5.3 and Appendix D.5, to evaluate robustness beyond the OOD action range used during post-tuning.

We consider two settings, Near and Far, both sampling actions outside the OOD-All post-tuning range along all three action dimensions. Far extends farther beyond this range than Near. We evaluate the existing Original and DG-ZA checkpoints without further training.

The action sampling ranges are:

ConditionΔx\Delta x (m)Δy\Delta y (m)Δψ\Delta\psi (rad)
OOD-All (post-tuning)[0.35, 0.50]±[0.15, 0.30]±[1.00, 1.50]
Near[0.55, 0.65]±[0.35, 0.40]±[1.55, 1.80]
Far[0.70, 0.80]±[0.45, 0.55]±[1.85, 2.10]

The 16-second Habitat-Sim results are:

ConditionModelLPIPS ↓DreamSim ↓FID ↓Support ↓
NearOriginal0.4740.36074.95877.365
DG-ZA0.4580.30150.47373.209
FarOriginal0.4690.33468.09876.210
DG-ZA0.4620.30255.70073.207

DG-ZA improves all four metrics under both extrapolation settings, reducing 16-second FID by 32.7% under Near and 18.2% under Far relative to Original.

We additionally aggregated the state–action verifier's outputs over the complete 16-second autoregressive rollouts. 99.11% and 99.17% of evaluated state–action pairs under Near and Far, respectively, were classified as OOD (q<0.5q < 0.5). For reference, the Habitat-Sim state–action verifier achieves 89.70% accuracy and 0.898 AUROC on its original verifier test distribution, as reported in Appendix C.2. The original verifier test set includes challenging state–action mismatches constructed by pairing states with actions from different time steps or trajectories, which may partly account for its imperfect classification accuracy.

These findings show that the verifier continues assigning low validity to extrapolated actions and that DG-ZA maintains improvements in long-horizon rollout stability beyond the action range used during OOD-aware post-tuning.

To examine sensitivity to incorrect validity predictions, we also conducted a verifier score inversion experiment, reported in Appendix G.3. We replace the estimated validity score qq with 1−q1-q during inference while keeping the trained DG-ZA checkpoint unchanged.

We report the following 16-second results under Habitat-Sim OOD-All actions:

Metric ↓Normal verifierInverted verifier
LPIPS0.4540.472
DreamSim0.2940.343
FID27.0262.03

Inverting the verifier score degrades all three metrics, demonstrating that the validity-dependent gating behavior is sensitive to incorrect validity assignments. This reinforces the importance of reliable state–action validity estimation for maintaining rollout stability.

Manuscript revisions

We have expanded the discussion of action controllability and downstream utility in Appendix H, and incorporated the counterfactual action-response experiment in Appendix G.2. We have also added action extrapolation results to Section 5.3 and Appendix D.5, and the verifier score inversion analysis to Appendix G.3.

AuthorsOpenReview ↗

Gentle Follow-up on Author Response

Thank you again for your careful review and constructive feedback.

As the discussion period draws to a close, we would like to briefly follow up on our rebuttal. We have provided additional counterfactual action-response, action extrapolation, and verifier sensitivity experiments, along with clarifications regarding the scope and contributions of our work.

We hope these additions have addressed your main concerns and would appreciate any further feedback or clarification requests. If you find that the new evidence sufficiently addresses your concerns, we would also be grateful for your consideration of an updated rating.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Our Rebuttal and Manuscript Revisions

We sincerely thank the reviewers and Area Chair for their time and constructive feedback. We have carefully considered the concerns raised during the review process and substantially revised the manuscript, incorporating additional experiments, statistical analyses, and clarifications.

As the discussion period draws to a close, we would like to summarize the main additions and how they strengthen the evidence for our contributions.

1. Generalization beyond the anticipated OOD action range (Section 5.3; Appendix D.5).

We conducted new Near and Far action extrapolation experiments using action magnitudes beyond those encountered during OOD-aware post-tuning, without retraining either model. DG-ZA consistently improves all four evaluation metrics, reducing 16-second FID relative to Original by 32.7% (Near) and 18.2% (Far). These results demonstrate that the improvements are not restricted to the exact action ranges used during post-tuning.

2. Statistical reliability of the improvements (Appendices D.4 and E.5).

We added paired-bootstrap 95% confidence intervals using 1,000 bootstrap replicates for the Habitat-Sim and WorldGym results. These analyses strengthen the statistical evidence for the observed OOD robustness improvements.

3. Action responsiveness versus long-horizon robustness (Appendix G.2; Appendix H).

To address whether improved visual metrics come at the expense of action-conditioned predictions, we conducted a one-step counterfactual action-response experiment with paired simulator ground truth. DG-ZA achieves positive action-response scores under both ID and OOD actions, demonstrating that its predictions remain responsive to changes in commanded actions. Although Original achieves higher scores in this one-step experiment, DG-ZA provides substantially better long-horizon perceptual alignment under OOD-All actions, improving all metrics relative to Original.

We have clarified this distinction and explicitly acknowledge that improved perceptual stability does not establish accurate extrapolation of arbitrary unsupported dynamics.

4. Further analysis of conservative collapse and verifier sensitivity (Appendices G.1 and G.3).

We added a support-distance re-entry analysis to quantitatively characterize candidate conservative-collapse events across different OOD perturbation strengths. We also conducted a verifier score inversion experiment, showing that incorrect validity assignments degrade rollout performance. These results further support the relevance of the identified failure patterns and the role of validity estimation in our mitigation approach.

5. Clearer novelty, positioning, and limitations (Related Work; Sections 4–5; Appendix H).

We expanded the discussion of MOPO, MOReL, DAgger, and RAE-NWM, clarified the architectural motivation for dual validity-aware gating, and strengthened the interpretation of the ablations. We also explicitly distinguish our goal of mitigating degeneration under unsupported actions from the more challenging task of faithfully predicting arbitrary unseen dynamics, while discussing downstream planning as future work rather than a demonstrated outcome.

Overall, we believe these additional experiments and revisions substantially strengthen the empirical support for our central contribution: identifying unsupported actions as an important source of autoregressive rollout degeneration and demonstrating that validity-aware action gating can mitigate this instability across navigation and robotic manipulation settings.

We respectfully hope that the reviewers and Area Chair will consider the additional evidence and revised manuscript in their final assessments.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.