acceptodds
Under review as a conference paper at ICLR 2027

When Should a World Model Trust Its Imagination? Local Empirical Support Predicts Recursive Rollout Failure

Abstract

World-model planners recursively evaluate imagined states that may drift into regions weakly supported by the data used to train the model. We ask whether local empirical support at an imagined state predicts rollout-prediction failure several steps later, beyond evidence that the model is already struggling at the current step. Using average -nearest-neighbor distance in learned latent space as a simple support probe, we find that local support adds predictive information beyond current one-step error, rollout position, and latent norm across multiple future offsets. The relationship transfers across five state-based MuJoCo tasks, visual inputs, deterministic residual dynamics, and a Dreamer-style recurrent state-space model (RSSM). It is not captured by the tested measures of global shift magnitude and remains partially informative after conditioning on transition disagreement and Mahalanobis distance, although the residual gain is much smaller in Hopper than in HalfCheetah. The most consistent pattern is geometric. Imagined states associated with future failure lie farther from training support across all dense state-based audits and every RSSM seed. Control experiments show that this diagnostic does not automatically yield a useful intervention. A support score can become more predictive while a fixed support penalty becomes worse, and percentile calibration can weaken an effective raw penalty. Overall, local empirical support is a reproducible forward-looking diagnostic of recursive rollout-prediction failure, while the appropriate control response must be validated separately.

Primary area:
reinforcement learning
Keywords:
world models, model-based reinforcement learning, uncertainty estimation, latent representations, model reliability, out-of-distribution generalization, recursive rollouts, nearest neighbors

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fp3UOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The authors raise the question of whether the average kNN distance of an imagined latent state to the world model's training latents predicts recursive rollout failure in future steps. They define failure as the rollout RMSE at t+5 exceeding the 80th percentile. The test is whether kNN distance raises AUROC over a baseline of rollout position, current one-step prediction error, and latent norm.

They open the paper with two claims:

  1. Local support is forward-looking beyond the current error and
  2. Diagnostic transfer does not imply intervention transfer
Critical strengths
  • Multi-Seed controls for the WM
  • Honest reporting: The authors report issues and limitations honestly
  • Supplementary code is reproducible
  • The paper's findings can be useful. Validating a reliability score separately from the control response built on it is sensible, practical evidence
Critical weaknesses
  • MW1: The baseline's "current one-step error" is computed from the true state at each step, not from the imagined rollout. In HalfCeetah, it is uniformative on its own: standalone AUROC. The HalfCeetah baseline (AUROC$ \sim $ 0.6) is therefore mostly rollout position ( $\sim $0.57). The current recursive rollout error, the natural sign that the model is "already struggling," sits in the same rollout table but is never used as a predictor. It is an oracle, exactly like the one-step error the paper includes to make the test conservative. Without it, the papers' first claim that kNN adds information beyond the current error remains untested. Please add the current recursive error to the baseline and report the KNN increment with and without Mahalanobis distance and disagreement. This should be done for every seed and extension (please refer to Q1).
  • MW2: Section 5.2 says the horizon sweep "directly tests" whether the kNN effect is merely contemporaneous. It cannot. A rollout that has already drifted away from the training data tends to stay there, so a signal that only measures current drift would also improve prediction at every future offset. The HalfCeetah gains are consistent with a simpler explanation: They barely change with the offset. To show that kNN anticipates failure, the authors could either control for the rollout's current error or predict how much the error will grow between now and the next step, rather than its level. Or test whether kNN identifies which states that have not yet failed will fail. As this section is the main evidence for the paper's central claim, it belongs in the main text, not only in the appendix.
  • MW3: That imagined states far from the training data later fail to restate the compounding error and model-exploitation premise that the introduction treats as known.
  • MW4: The introduction never specifies what a planner should do upon receiving a warning, and the study then tests a single, fixed additive penalty. "Diagnosis does not imply intervention" is therefore shown for a single response family.
  • MW5: The main text does not carry the evidence. It has two figures and two tables. Section 5.3's 160/160 counts and section 5.4's conclusions cite no table or appendix. Additionally, 5.4 reports no Number. The world model design is uncited, the classifier behind every AUROC is never named, and the reward model and planner appear in Section 3.4 without any motivation (Details are unmentioned; from this reviewer's perspective, they are referred to in the appendix). The fixed-horizon setting, the 80th-percentile threshold $ \lambda = 0.5 $, 90th-percentile scale matching, and everything else are simply introduced and not supported by any cited scientific backing or ablation study.
  • MW6: The Authors' names and numbers are carefully scoped to prevent overclaim. This, however, generally limits the interpretation of results at other parts, for example, the Hopper/HalfCeetah gap. The authors call the gap but never explain it, even though the evidence (MW1) could do so.
Questions for authors
  • Q1: Please report kNN and Baseline + current recursive rollout error, with and without Mahalanobis and disagreement. Please do so for all seeds and extensions, and clarify if the "beyond current error" claim survives.
  • Q2: Could the authors please state the classifier that produced the reported AUROCs? It should be stated, and a flexible baseline added (gradient boosting or one-hot rollout position) so that kNN cannot compensate for a misspecified linear model.
  • Q3: How does the horizon sweep separate anticipation from persistence? Predicting the onset of failure among states whose current recursive error is still below the threshold would directly test the paper's own motivation.
  • Q4: How sensitive are the results to the 80th percentile threshold, k, and $\lambda $ under the corrected support reference?
Reviewer_HJ2VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

can local empirical support in a world model's learned latent space predict future recursive rollout failures is the central research questions

this is studies empirically - in some instances it helps in others it doesnt

Critical strengths

evaluates multiple environments, dynamics perturbations, rollout horizons, and architectures, with seed-level replication and additional controls for data leakage

Critical weaknesses

Limited novelty

Questions for authors

Does kNN distance predict future failures before significant prediction errors have already happened?

Would the results also hold for world models trained and evaluated using learned policies rather than random action

Reviewer_fqvnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper looks at representational methods for detecting the quality of future "imagined" states. The authors use a KNN distance between future states imagined by a model and attested states in the training data as a kind of early waring signal for whether or not models' hypothesised future states are likely to be valid. They show this kNN based method is able to predict whether or not a model's roll-out will ultimately succeed better than next-step error alone. In some preliminary experiments they show this score can in some context be added as an objective with moderate gains.

Critical strengths

This paper looks at a broadly interesting question: how can we tell if a model's future predictions are likely to be right. Additionally they do so across multiple simulated domains, and make an effort to offer controls and baselines where applicable.

The authors are also forthright about some of the limited novelty in this work; making clear the core kNN method is not original to the work here.

Critical weaknesses

The paper has limited novelty, it uses an existing metric across synthetic toy tasks to show modest predictive power. Additionally experiments using this metric as an objective have limited success.

Additionally I believe the baselines used are not quite appropriate. The authors compare agains next step error, when a more appropriate metric might be aggregate error. It's possible that a trajectory is cumulatively wrong, but the next step is only a minor further error. The kNN method would be able to detect the cumulative errors of the preceding context. So while it may beat next step error it may not beat accumulated error.

The paper is quite challenging to read. The introduction does not clearly introduce the domain or architectures. Additionally I don't believe the classifier architecture is ever described.

Questions for authors

Have you compared against accumulated error rather than next step error?

The gains against existent baselines seem marginal, could you be clearer about the statistical testing that supports your novelty claims?

What is the broader applicability of your central findings?

What is the architecture of the classifier used.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.