acceptodds
Withdrawn from ICLR 2027

Defect Responses in Visual World Models: Long-Horizon Propagation, Gradient Interaction, and Planning Outcomes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual world-model predictors are trained on local latent targets, whereas planners rank action sequences through composed predictions. Defect response is the separation between two rollouts that share future actions: one starts from the observed next latent, while the other replaces its non-action channels with the model prediction. We ask whether attenuating this response over long horizons improves planning. Defect-Response Fan (DRF) penalizes the terminal response and assigns predictor-parameter credit only through the injected prediction. A full-gradient all-suffix control (FG-AS), inspired by ACPC's same-action pairing, penalizes every suffix response and differentiates through both branches. We freeze every component except the predictor and compare both objectives with Native one-step training on PushT and PointMaze using five paired seeds, a training-inaccessible candidate bank, and matched 1,200-update and 1,800-second total-wall views. Under matched wall time, FG-AS has lower mean-regret point estimates than Native by 4.57% on PointMaze and 7.03% on PushT, despite reaching fewer updates; at 1,200 updates, the direction reverses on both tasks. All paired regret intervals include zero, so the prespecified cross-task, dual-budget confirmation gate fails. Across 40 method–Native contrasts, three response measures attenuate in 23–25 cases, but their Spearman correlations with true-regret improvement are -0.148, 0.052, and 0.032, with cluster-stratified intervals spanning zero. DRF and FG-AS retain 24.6% and 17.7% of Native optimizer throughput. In this setting, the objectives measurably alter response propagation, but response attenuation is not established as a reliable surrogate for planning quality.

Primary area:
applications to robotics, autonomy, planning
Keywords:
visual world models, model-based planning, defect-response training, long-horizon prediction, gradient interaction, DINO-WM, robotic manipulation, navigation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Y4g8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies whether planning can be improved by training world models to be less sensitive to their own prediction errors. The authors introduce two auxiliary losses that compare simulated futures starting from the real next state and the model's predicted next state, under identical subsequent actions. They fine-tune an existing DINO-WM predictor using these losses and evaluate whether it ranks candidate action sequences more accurately. Although the losses alter how prediction errors propagate, the resulting planning improvements are inconsistent and depend on the training budget.

Critical strengths
  1. The paper directly tests whether making a world model less sensitive to its own prediction errors leads to better action selection, rather than assuming that it does. The authors state that the test was specified in advance (Pages 2 and 6), and they clearly report the negative result, noting that “Every FG-AS regret interval includes zero” (Page 7). This makes the finding more credible, and it shows why it is important to test whether this measure is useful before using it as a training objective.

  2. The comparison is carefully controlled. All methods start from the same DINO-WM predictor and evaluate the same candidate action sequences (Table 1, Page 6), and the selected sequences are scored with the true simulator (Page 7). The authors also report results both at an equal number of training updates and at equal wall-clock time, along with the cost of each loss (Table 6, Page 15). This is useful because it shows how the conclusions change with the training budget, which would be easy to miss if only one comparison were reported.

Critical weaknesses
  1. The better results at equal wall-clock time may simply come from FG-AS training for fewer steps, rather than from the auxiliary losses. Table 2 (Page 7) shows that Native performs worse as training continues: from 1,200 updates to 1,800 s, its regret rises from 3.124 to 3.309 on PointMaze and from 11.570 to 13.088 on PushT, while FG-AS changes much less (3.228 to 3.158 and 12.485 to 12.168). Native after 1,200 updates also has the lowest regret of all six method and budget entries in Table 2, on both tasks. Within 1,800 s, Native runs about 7,748 updates and FG-AS about 1,289 (Table 6, Page 15). This makes the main positive result less convincing, since FG-AS may only look better because it avoids the drop in performance that comes with longer training. The one paired interval that excludes zero, a 5.00-point success gain on PointMaze at 1,800 s (Page 7), follows the same pattern, as Native's success falls from 66.46 to 62.92 while FG-AS goes from 67.50 to 67.92. This matters because the result reverses at equal updates, and none of the regret differences is statistically clear: every FG-AS regret interval includes zero (Page 7).

  2. The paper does not explain why reducing the model's sensitivity to its own prediction errors does not reliably improve planning. Across 40 comparisons with Native, the response is reduced in 23 to 25 cases, but regret improves in only 18, and both happen in only 10 to 12 (Section 5.2, Page 8). The authors acknowledge that their gradient diagnostics establish “scale mismatch and occasional local opposition, not a cause of planning outcomes” (Page 8), that the mechanism analysis “remains descriptive”, and that their comparison cannot identify routing as the cause of a planning difference (Page 9). The broader idea that better internal model measures do not always lead to better decisions is already known, as the introduction itself notes (Page 1). With two tasks, one architecture and five seeds (Section 7, Page 9), it is also hard to tell whether the method has a small benefit or none at all. Since the negative result is the paper's main contribution, I would have liked a clearer explanation of why it happens and what new insight it provides.

Questions for authors
  1. (Clarification) Could you show how Native's regret changes throughout training? In Table 2, Native performs better after the same number of updates, but FG-AS performs better after the same training time, mainly because Native's regret increases with further training while FG-AS improves slightly. The training curve would show whether the improvement comes from the auxiliary losses or simply from differences in training behaviour. The paper notes that DRF shows the same directional split between the two budgets (Page 7).

  2. (Clarification) Could you show an example from Figure 4 (Page 8) where the response is reduced but regret is higher than Native's? Figure 3 (Page 8) shows representative roots with their true regret and success, but not the predicted costs of the candidates or how each model ranks them. Showing these for one root from such a contrast might help explain why a lower response does not lead to better choices.

Reviewer_6wUiOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether attenuating multi-step trajectory separation in visual world models improves downstream planning quality.

Critical strengths

None

Critical weaknesses

While I can believe that the authors checked the AI generated text, the text is almost impossible to follow.

Almost every paragraph is packed with multi-hyphenated compound phrases like "equal-forward-value gradient-routing probe," "cluster-stratified intervals," "training-inaccessible candidate bank," and "residual-anchored, same-action response." as a non properly guided LLM would do. Similarly, the authors write with extreme defensive precision, which is typical of LLM as well, and make it's hard to understand the core contributions (see terms like "prespecified cross-task, dual-budget confirmation gate" and "inferentially eligible")

Section 3 introduces projection operators (πx\pi_x), shift operators (S(C,z)S(C,z)), rollout operators (Gθ,aG_{\theta,a}), and stop-gradient functional snapshots (Rt,e(u;sg(θ))R_{t,e}(u;\text{sg}(\theta))) to describe something that is conceptually straightforward (or at least this is what I was able to parse): "take a real frame and a 1-step predicted frame, feed both the exact same future actions, and penalize how far apart their trajectories drift.

Questions for authors

I'd highly recommend to fully rewrite the paper by hand as the current state makes it impossible to parse.

Reviewer_NqodOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to reduce how world model prediction errors propagate over longer horizons. Starting from DINO-WM, it generates two autoregressive rollouts under the same actions, one initialized with the model's predicted next latent state and the other with the ground-truth next latent state. The proposed losses explicitly minimize the discrepancy between these rollouts. Experiments are conducted on PushT and PointMaze.

Critical strengths
  • The supplementary materials are extensive, including experimental code and code for reproducing the figures.
Critical weaknesses
  • Limited novelty. The general idea of reducing compounding errors in autoregressive rollouts is not fundamentally new. While the paper introduces a specific defect-response formulation and gradient routing strategy, its novelty over existing approaches is not sufficiently established.
  • Insufficient comparisons with closely related methods. Training world models to reduce multi-step prediction errors or improve rollout stability has been extensively studied [1-3], but stronger comparisons under a matched experimental protocol are missing.
  • Limited experimental validation. Experiments are conducted only on PushT and PointMaze, which may not be sufficient to assess the proposed method's generality. More challenging manipulation benchmarks, such as LIBERO, RoboCasa, or OGBench, would provide stronger evidence. Moreover, closed-loop planning evaluation, commonly adopted in the field, is missing for the proposed methods. The authors should evaluate on at least one such benchmark, preferably including longer-horizon tasks. The reported planning improvements are also not statistically conclusive.
  • Writing and organization could be significantly improved. The paper is difficult to follow, partly due to excessive technical and experimental details introduced too early. For example, details such as hyperparameters, random seed protocols, and statistical testing criteria in the Introduction would be better placed in the Experiments section or supplementary materials. The paper should provide a clearer high-level explanation of the method and its motivation before introducing implementation details.

[1] What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?

[2] V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning

[3] Towards Unraveling and Improving Generalization in World Models

Questions for authors

See the "Critical Weaknesses" section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.