acceptodds
Under review as a conference paper at ICLR 2027

Support-Aware Shielding Restores Safety for Predictive World-Model Control under Shift

Abstract

A predictive world model is supposed to let an operator ask “what if” before committing to a change, and that promise rests entirely on the model knowing when it is wrong. We study what becomes of the promise when the world moves. Working in a setting where every planning decision can be replayed against a full-information oracle, we find a pattern that repeats across every model we train: the planners that do best are the planners that are most dangerous. The models that extract the most benefit under familiar conditions are precisely the ones that breach the operator's safety constraints once a crowd forms, and the uncertainty intervals a shield would have to trust quietly stop covering the truth exactly when they are needed. This failure is invisible from forecasting accuracy alone: models that are nearly indistinguishable on held-out prediction error behave entirely differently once their forecasts are allowed to drive actions. We then show that the remedy is not a better forecaster. It is a planner that keeps asking whether it still recognises the world. A controller that re-plans as it goes, replays its own model against what it has just observed, checks whether its forecast has wandered outside the region its training data covered, and declines to act when either test fails, recovers nearly all of the lost safety while surrendering only a small and bounded share of the achievable benefit. We prove two statements that explain the exchange - one tying how reliably shifted situations are recognised to how often constraints can be breached, one showing that the cost of declining to act is paid in proportion to how often it is declined - and we machine-check both. The lesson we draw is that safety under distribution shift is more cheaply bought with an online admission of ignorance than with a more confident model.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
World Model, JEPA, Out of Distribution, Deep Learning Theory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9X6rOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors acknowledge that the drafting, writing, and copy-editing of this paper involves the use of AI tools. This is not a problem per se, but, as a reviewer, I find the resulting manuscript extremely hard to read, understand, or interpret.

This is my (very rough, possibly mistaken) understanding:

  • In a simulated mobile network scenario of n cells, each cell is associated with 3 parameters. Every 5 minutes, every cell reports some indicators, two of which are relevant for safety (the "drop rate" and the "handovers"). A "plan" is a choice of these three parameters for every cell at every time step, and there's a baseline plan A_0. A plan is useful relative to the improvements of the indicators over A_0, and is unsafe if its worst drop rate or its worst handover are worse than the baseline.
  • The goal is to propose a planner that is more useful and more safe than existing models, especially ood.
  • The authors claim that a planner that re-plans hourly and abstains when necessary is more useful and safer ood.
Critical strengths
  • I do not understand the paper with enough detail to point at strengths.
Critical weaknesses

The manuscript is extremely hard to follow, because the main text does not define most of its objects, including the experimental scenario.

  • The intro does not explain what is trained and what is not, nor on what data. It does not say what the seeds vary, it does not mention what the type of models are, what is the baseline. It introduces generic terms without defining or explaining them (e.g., "a model removes", "oracle caution", "demand", "environment model", "feature interface", "first regime", "open-loop planner", ...). The same comments apply to the figures.
  • All the main terms ("admission ceiling", "handover margin", "time to trigger", the quality indicators, the safety indicators, J, ...) are introduced without any definition or explanation (what is their type? what are their units? what do they mean? ...). The same applies to the environments and the features.
  • The values of the baseline A_0 is not present in the main text.
  • The utility and safety test are not defined in the main text.
  • It is not clear what the data is.
  • ...

Therefore, I opt to flag this for ethics review (unprofessional behavior).

Questions for authors

N/A.

Reviewer_m2rBOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper considers the safety of actions driven by a learned predictor under distribution shifts. Safety is defined relative to a baseline policy and is closely related to notions of harm, e.g., Richens et al (2022). The domain is simulated mobile-network traffic steering, with a concrete specification. The predictor is trained to output a calibrated distribution of environment indicators. The quantiles of those distributions give a probability of unsafe behaviour which can be expected to be calibrated in-distribution but inaccurate out-of-distribution. The authors then propose a controller that stops an agent's choice of action once a residual check or a support check says the current environment step is out-of-distribution.

Richens, Jonathan, Rory Beard, and Daniel H. Thompson. "Counterfactual harm." Advances in Neural Information Processing Systems 35 (2022): 36350-36365.

Critical strengths

The authors highlight that good prediction/forecasting behaviour is not equivalent to good planning behaviour (on top of those predictions). This insight is very valuable, and makes an interesting prediction/safety tradeoff. The control environment is also well-motivated and makes the problem apparent as the simulator defines an oracle planner.

Critical weaknesses

In my view, the lack of mathematical formalisation is a weakness. I also believe that it would be useful to place the contribution in a more general framing, applicable beyond the set of applications considered by the authors.

In the more general case it strikes me that deciding when in instance is out-of-distribution and what mitigation strategies are generally applicable is difficult and not discussed by the authors. For instance, the two strategies proposed by the authors: self-consistency replay and a predicted-state support test, are somewhat heuristic and maybe specific to the environment in which the methods are tested. It would be useful for readers to understand how widely applicable these techniques are.

Decision-making out-of-distribution has a large literature, and it is not clear how the author's work relates to prior art. For example, "When to Act, Ask, or Learn: Uncertainty-Aware Policy Steering" by Yuan et al. is very related.

Minor: The text has many elements characteristic of LLM written papers. This is ok but it can sometime be difficult to grasp the main contribution because the text doesn't convey the intuition and relevance behind the main results as well as a human might have.

Questions for authors

Please consider writing key notions in the paper more formally. For example the definition "a plan is unsafe if its worst-case-over-time value on either regresses against A0 by more than a tolerance" is ambiguous. The following sentence then mentions regret without definition. The problem formulation is specific to one environment, e.g., see how the action space is defined. Could it be generalised to a class of environments? What should the forecaster predict for deciding if an instance is ood?

The authors mention "What we contribute is not a new estimator in either family but the observation that, inside a control loop, a coarse support test used to abstain recovers more safety than the calibrated intervals themselves can supply once shifted." I am not sure I found evidence of this claim in the experiments, did you compare with conformal prediction methods or re-calibration methods on shifted distributions? It is not clear to me that you can recover safety by re-calibrating the prediction interval?

Reviewer_wFmcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to make AI controllers safer with learned world models under distribution shift. The main idea is that a controller should continually check whether its predictive model remains trustworthy and fall back to an existing policy when those checks fail. In simulated mobile-network traffic management, it finds that degraded uncertainty calibration allows prediction-based safety shields to approve harmful interventions. The proposed controller replans periodically and falls back to baseline settings when recent prediction errors or forecasted conditions indicate that the model may be unreliable. Across the studied shifted scenarios, the combined approach reduces unsafe scenarios, while retaining useful performance; though these empirical observations are limited to simulation.

Critical strengths
  1. The paper addresses an important, realistic problem: how to make AI controllers safer under distribution shift. The proposed approach is simple and practical.
  2. The combined controller substantially improves safety in the tested simulations while retaining useful performance.
Critical weaknesses
  1. The main critical weakness is that all experiments use simulated data. The effectiveness of the approach on real networks remains untested.
  2. The main comparison changes replanning and abstention together, so it does not establish how much improvement comes from the proposed support checks.
  3. Equation 5 appears inconsistent with the stated direction convention: increasing call-drop rate produces a negative violation score. Is this a notation error or does it affect the implementation.
Questions for authors

See Weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.