acceptodds
Under review as a conference paper at ICLR 2027

WISH: Margin-Free Latent Control Barrier Functions via World-Model Imagination

Abstract

Safety filters learned in world-model latent spaces commonly use binary failure labels to define unsafe states. However, their performance depends on the choice of a margin function that grades safety over nominally safe states, which is difficult to obtain in latent spaces. We propose WISH, a latent filter that measures the robustness of a latent state by the minimum cumulative disturbance effort required to reach a learned failure set against a fallback policy. Theoretically, for the exact modeled dynamics and value function, we prove that the resulting filter prevents failure when the initial state has sufficient disturbance resistance and the cumulative disturbance effort remains below a given budget. We also derive finite-horizon failure-probability bounds under stochastic imagination. In practice, WISH trains the latent safety value function entirely in the imagination of the world model by a minimum cost effort adversary against a fallback policy. Across four pixel-based continuous-control tasks, WISH achieves better safety-performance tradeoffs than existing latent filtering baselines.

Primary area:
reinforcement learning
Keywords:
safety, latent control barrier function, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_BDXoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes WISH, which addresses safety filtering from images when binary failure labels are available but a meaningful continuous safety margin is not. It defines robustness as the minimum cumulative disturbance effort needed to reach failure under a fallback policy, extending LEAP to discrete-time world-model dynamics. For the exact value and admissibility condition, it proves safety under a cumulative disturbance budget and derives finite-horizon failure-probability bounds. Its implementation learns an approximate value through adversarial RL in imagination and reports improved safety-performance tradeoffs on four simulated visual-control tasks.

Critical strengths

S1. A useful adaptation of minimum-effort safety filtering to latent world models. Building on LEAP’s minimum-effort value and cumulative-budget safety principle, WISH instantiates disturbances through the world model’s noise channel and measures their effort using shifted negative log-likelihood. This provides a dynamics-dependent safety scale using binary failure labels, without requiring a hand-crafted margin or an explicit physical disturbance model. The paper develops the corresponding discrete-time filtering condition and finite-horizon failure-probability bounds under exact modeled dynamics and values. The resulting formulation provides a principled basis for ranking states by their modeled disturbance resistance, with Figure 1 illustrating the potential usefulness of this distinction.

Critical weaknesses

W1. The mechanism enforcing successful attacks is underspecified. Section 4.3 states that the adversary is penalized for failing to reach the failure set, but the explicit per-step reward is only −ξ(ϵt)-\xi(\epsilon_t). Appendix C.3.1 describes the reachability classifier, two value heads, and critic polishing without specifying how this penalty enters adversary optimization. Under the stated reward alone, choosing zero disturbance achieves maximal return without requiring failure. It is therefore unclear how the implemented training supports interpreting the learned value as the minimum cost of successful attacks in Eq. (8). The authors should specify the complete adversary objective, treatment of unsuccessful rollouts, and targets for both critic heads, explaining how these prevent the trivial zero-disturbance solution.

W2. The sources of the empirical gains are only partially isolated. The shared-fallback experiment in Figure 5(a) supports an advantage beyond fallback improvement on Cheetah. However, Figure 5(c) replaces the HJ margin on Waiter without clearly stating whether the fallback is held fixed. Consequently, the evidence separating improvements in the value/filter from improvements in fallback quality remains limited outside Cheetah. The authors should clarify the Waiter ablation and provide a targeted shared-fallback comparison on Walker or Waiter.

Questions for authors

Q1. What are the complete adversary reward and regression targets for both critic heads? How are unsuccessful or truncated attacks treated, and what prevents the zero-disturbance solution under the stated reward alone? Please also clarify Figure 6’s return definition and accumulation horizon: Cheetah returns reach approximately −410, whereas a 15-step sum of the stated disturbance costs cannot exceed 240.

Q2. On held-out states, how frequently do learned attacks reach failure, and how well does J^dep\widehat J_{\mathrm{dep}} reflect their undiscounted disturbance costs?

Q3. Does Figure 5(c) hold the fallback fixed or retrain it with the new margin? If it is retrained, an additional shared-fallback comparison on Walker or Waiter would help establish whether the Cheetah finding extends beyond that task. Please also specify the threshold ranges used to construct the frontiers.

Q4. Under the exact-model/value assumptions, can the authors illustrate when Corollary 1 provides an informative bound while satisfying D≤Jadvπfb(z0)D\leq J_{\mathrm{adv}}^{\pi_{\mathrm{fb}}}(z_0)? How does restricting the training adversary to ∣ϵ∣∞≤1|\epsilon|_\infty\leq1 affect the learned value’s interpretation relative to the unbounded Gaussian noise assumed in the corollary?

6 comments
AuthorsOpenReview ↗

Response to Reviewer BDXo (1/3)

We thank the reviewer for raising valuable comments. We are excited that the reviewer identified that WISH provides "a useful adaptation of minimum-effort safety filtering to latent world models" and defines a "principled basis for ranking states by their modeled disturbance resistance". We believe that WISH takes a significant step toward constructing better robust safety filters in latent space. We try our best to resolve all raised concerns in the responses below.

 

W1, Q1: Can you clarify how the adversary's optimization problem is solved?

In the updated version of the manuscript, we have refined our explanation on how the adversary's optimization problem is solved to enhance readability. Specifically, we have elaborated on the mechanism that guides successful attacks:

We use a discounted disturbance-cost objective as a heuristic surrogate for this constrained problem, where the adversary is encouraged to reach F^\hat{F} because it tries to maximize a nonpositive per-step reward (negative of the disturbance cost) and the episode terminates when F^\hat{F} is reached. The adversary’s policy never collapses to the trivial zero-disturbance policy in practice due to entropy regularization, so it is incentivized to reach ˆF to avoid accumulating infinite cost. Discounting prevents infinite costs which breaks the equivalence and is a limitation of this simple surrogate, which could be addressed with more principled minimum-cost reach-constrained methods such as Liao & Liang (2021); So et al. (2024a). See App. C.3.1 for full details on the adversary training.

What happens to unsuccessful attacks? Attacks that are unsuccessful at the end of the rollout (and thus truncated) bootstrap their value and accumulate potentially infinite costs, in comparison to successful attacks that terminate the episode and have finite cost. This incentivizes the adversary to perform successful attacks. However, discounting means that an unsuccessful attack could have a lower cost than a successful attack. We already acknowledge this in our Limitations section.

Despite this approximation, in practice the learned attacks nearly always reach failure (see answer to Q2).

 

Q1: Please also clarify Figure 6's return definition and accumulation horizon.

Thank you for catching this. We were previously plotting the mean of the value target which includes a bootstrap and thus corresponds to the cost of the whole attack until failure rather than the cost of the 15-step window. We have corrected Figure 6 in the revised pdf to directly the following directly measured quantities:

  • the undiscounted cost ∑t12∥ϵt∥2\sum_t \tfrac{1}{2}\|\epsilon_t\|^2 accumulated until failure, averaged over the attacks that reach failure;
  • the fraction of starts from which the attack reaches failure
  • the fraction of starts that fail under zero adversary

 

Q2: On held-out states, how frequently do learned attacks reach failure?

Nearly always. The adversary reaches failure in over 99% of the held-out states over all tasks.

For each task we take 400 held-out states (i.e., from episodes the world model never trained on) for each of the 3 WISH seeds, and roll out the learned adversary against the fallback policy for 300 steps in imagination.

TaskAttack reaches failure
Cartpole100.0% (1200/1200)
Cheetah99.8% (1197/1200)
Walker99.9% (1199/1200)
Waiter99.3% (1192/1200)

We have added this discussion and table to Appendix C.3.5 in the revised pdf.

 

AuthorsOpenReview ↗

Response to Reviewer BDXo (2/3)

Q2: How well does J^dep\widehat{J}_{\mathrm{dep}} reflect the undiscounted disturbance costs on held out states?

Fairly accurately.

The learned attack reaches F^\hat{\mathcal{F}} from 99.3–100% on held-out states, while the Spearman correlation between J^dep\widehat{J}_{\mathrm{dep}} and the undiscounted cost ranges from 0.66 to 0.97 over tasks on held out states. On latents from the dataset, the Spearman correlation increases to 0.81 - 0.97. This indicates that there is a small generalization error on held-out states, but otherwise the learned J^dep\widehat{J}_{\mathrm{dep}} tracks the disturbance costs well.

  • Held out states: latents that were not in the training dataset
  • Train states: latents from the dataset used to train the world model, adversary and values.
TaskHeld-out statesTrain states
Cartpole0.660.88
Cheetah0.680.81
Walker0.840.82
Waiter0.970.97

We have added this discussion and table to Appendix C.3.5 in the revised pdf.

 

W2, Q3: In Figure 5c, does HJ+VDHJ + V_D use the same fallback as WISH?

No, HJ+VD\mathsf{HJ + V_D} doesn't use the same fallback as WISH. In Figure 5c, HJ+VD\mathsf{HJ + V_D} retrains its own fallback policy against the new margin VDV_D, so that comparison changes the margin and the fallback together.

As suggested, we ran a shared-fallback comparison on Waiter and added it to Figure 5c in the revised pdf as HJ+WISH  πfb\mathsf{HJ + WISH\; \pi_{\mathrm{fb}}}, which is HJ(GP)\mathsf{HJ (GP)} but with the fallback policy initialized and frozen to WISH's fallback πfb\pi_{\mathrm{fb}}.

We make the following observations:

  • HJ+WISH  πfb\mathsf{HJ + WISH\; \pi_{\mathrm{fb}}} still improves over HJ(GP)\mathsf{HJ (GP)}, confirming that the WISH fallback is a large part of the gain on Waiter.
  • The remaining gap to WISH at matched performance shows that the minimum-effort value contributes beyond the fallback, consistent with the Cheetah result in Figure 5a.

What about HJ+VD+WISH  πfb\mathsf{HJ + V_D + WISH\; \pi_{\mathrm{fb}}}?

One can also consider an ablation where we use WISH's value VDV_D as the margin and fix the fallback to be WISH's fallback to try and "isolate" the HJ value backup versus how WISH learns the value. However, one can show that the WISH value is the solution to the HJ backup when using WISH's VDV_D as the margin and WISH's fallback (Appendix B.5). Thus, this variant does not "isolate" HJ value versus WISH value since the values coincide theoretically. We were able to confirm this empirically and found that the fronts basically coincided (Figure 8 in the revised pdf).

 

Q3: Please also specify the threshold ranges used to construct the frontiers.

For each method, we use 20 linearly spaced thresholds in [0,c][0, c], crossed with α∈{0.1,0.5,0.8}\alpha \in \{0.1, 0.5, 0.8\}. Ceiling cc is 1.05×1.05 \times the 99th percentile of that method's deployed value over safe latents from the world-model dataset, averaged over the three training seeds. For MPS, cc is 1.05×1.05 \times the 95th percentile of its margin, and there is no α\alpha. The resulting ranges are:

Main results (Fig. 4)

TaskWISH (DD on JJ)HJ (GP) (DD on VV)HJ (no GP) (DD on VV)MPS (margin mm)
Cartpole[0, 97.4][0,\ 97.4][0, 1.00][0,\ 1.00][0, 1.05][0,\ 1.05][0, 1.98][0,\ 1.98]
Cheetah[0, 205.9][0,\ 205.9][0, 0.97][0,\ 0.97][0, 1.05][0,\ 1.05][0, 1.70][0,\ 1.70]
Walker[0, 52.6][0,\ 52.6][0, 0.96][0,\ 0.96][0, 1.05][0,\ 1.05][0, 1.68][0,\ 1.68]
Waiter[0, 290.9][0,\ 290.9][0, 0.83][0,\ 0.83][0, 1.05][0,\ 1.05][0, 1.14][0,\ 1.14]

Ablations (Fig. 5)

PanelWISHWISH (HJ fallback)HJ (GP)HJ + VDV_DHJ + WISH πfb\pi_{\mathrm{fb}}
a. Cheetah[0, 205.9][0,\ 205.9][0, 203.3][0,\ 203.3][0, 0.97][0,\ 0.97]––
c. Waiter[0, 290.9][0,\ 290.9]–[0, 0.83][0,\ 0.83][0, 0.71][0,\ 0.71][0, 0.68][0,\ 0.68]

Dash means not applicable. Note that Fig. 5b has no threshold since it runs each fallback alone, without a filter. In addition, recall that WISH ranges are in units of disturbance cost and so differ across tasks; the HJ values are bounded by 1 (tanh), so their ranges are close to [0,1][0, 1]. We have added these tables and the rule above to Appendix C.3.4 (Tables 2 and 3) in the revised pdf.

AuthorsOpenReview ↗

Response to Reviewer BDXo (3/3)

Q4: Under the exact-model/value assumptions, can the authors illustrate when Corollary 1 provides an informative bound while satisfying D≤JadvπfbD \leq J_{\mathrm{adv}}^{\pi_{\mathrm{fb}}}?

Good suggestion. Our existing toy example provides a concrete illustration. Here, Jadvπfb(4)=25/2J_{\mathrm{adv}}^{\pi_{\mathrm{fb}}}(4)=25/2. Choosing D=25/2D=25/2 and N=10N=10, Corollary 1 gives

\leq 1-F_{\chi^2}(25;10) \approx 0.005346,$$ certifying a safety probability of at least **99.465% over N=10 steps**. **We add additional discussion on the informativeness of the bound from Corollary 1 in Appendix B.11.1**, and have added the following paragraph to the main text: > The informativeness of Cor. 1 depends mainly on the event inclusion in Thm. 1. For fixed D, the upper bound can become conservative over longer horizons N. We illustrate this on the toy example (App. B.11) in App. B.11.1, showing both an informative certificate over short horizons and this conservatism over longer horizons.   ## **Q4:** How does restricting the training adversary to $|| \epsilon ||_\infty \leq 1$ affect the learned value's interpretation relative to the unbounded Gaussian noise assumed in the corollary? Good catch! Indeed, from the perspective of Corollary 1, it is more accurate to use an unbounded action space for the adversary. The current $||\epsilon||_\infty \leq 1$ constraint can be interpreted as changing Corollary 1 to consider truncated Gaussian noise instead of standard Gaussian noise. To assess the effect of this implementation choice, we **rerun our experiments with an unrestricted adversary**. We find that **performance is essentially unchanged on all tasks** except for Waiter, where the method drops in performance relative to the $||\epsilon||_\infty \leq 1$ variant but still outperforms all the baselines (Figure 7 in the revised pdf, will replace Figure 4 after the updated ablations finish running). We therefore adopt the unrestricted implementation for the main results in the revised pdf. We are not yet able to explain why the $||\epsilon||_\infty \leq 1$ performs better only on Waiter. We leave this to future work.
Reviewer_gtQROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes WISH, a safety filter that measures how much disturbance is needed to cause failure in a learned world model. A state is considered safer when even the cheapest successful attack requires substantial effort. This gives a continuous safety measure using only binary failure labels. In practice, an adversary is trained to find the cheapest successful attack.

Critical strengths

The experimental results shown are very promising.

Critical weaknesses
  1. Does the practical algorithm being used actually inherit the safety certificate proven in theory? It seems like there is a gap between what is shown theoretically and what is implemented in the algorithm, where the substantial use of approximation and black box learning process undermines the theoretical result.
  2. There are different ways to replace binary safety labels, replace them with some version of probabilistic values, and avoid unavoidable failures in the future. This paper is particularly similar to Wang, Zhuoyuan, et al. "Myopically verifiable probabilistic certificates for safe control and learning." IEEE Transactions on Automatic Control (2026). If considering the practice of learning an adversarial for cheapest attack, it is also very similar to Hoshino, Hikaru, and Yorie Nakahira. "Physics-informed RL for maximal safety probability estimation." 2024 American Control Conference (ACC). IEEE, 2024. These works should be minimally put into discussion for similarity, if not as additional baselines.
Questions for authors

See Critical Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer gtQR

We thank the reviewer for raising valuable comments. We are excited that the reviewer identified that "the experimental results are very promising" and recognized our paper's presentation, informativeness and soundness. We believe that WISH takes a significant step toward constructing better robust safety filters in latent space. We try our best to resolve all raised concerns in the responses below.

 

W1: Does the implemented algorithm inherit the safety certificate proven in theory?

No. The implemented algorithm uses neural network approximations and approximate optimization, so it does not inherit the safety certificate established under the assumptions of our theoretical analysis. We already acknowledge this in the limitations section:

The exact guarantees do not hold for the learned approximations without additional verification (Zhang et al., 2023). The discounted adversary objective does not explicitly enforce successful failure, so even exact optimization of this surrogate need not correspond to the solution in (8).

We view the theory as a principled foundation for algorithm design as it establishes safety in the exact setting and motivates our practical approximations.

 

W2: There are existing works that make use of binary safety labels to avoid failures in the future

Thank you for pointing us to that work, it is indeed relevant. We have added it to our extended related work section, as it does use binary safety labels directly without assuming access to a margin function. However, that work considers continuous time SDEs, which differs from the discrete-time uncertain dynamics in this work. Moreover, that work proposes an RL algorithm that computes a finite difference estimate of a gradient on each step of the rollout, which is prohibitively expensive in our setting.

 

W2: If considering the practice of learning an adversarial for cheapest attack, it is also very similar to this paper.

To the best of our understanding, we were not able to find where it mentions learning an adversary for investigating robustness under adversarial attacks in that paper. Our understanding is that that work proposes to estimate the maximal safety probability by maximizing a reward function to find a policy that survives for N steps.

While both our work and that paper work with a sum-of-rewards RL problem, we propose to find an adversary that causes failure under the cheapest attack possible, whereas the proposed work find a policy that maximizes the probability of surviving for N steps.

Nevertheless, it is still somewhat related to our work, and we have added it to our extended related work section.

AuthorsOpenReview ↗

Followup with Reviewer gtQR

Thank you again for your valuable feedback. We understand that this is a busy period, and we sincerely appreciate your time.

With approximately 11 hours remaining in the author-reviewer discussion period, we would appreciate it if you could let us know whether our responses have adequately addressed your concerns. If so, we kindly ask you to reconsider your rating.

Thank you so much!

My concerns are mostly addressed. The score is raised to weak accept.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Ping to reviewers

We thank the reviewers again for their time and feedback! With 2 days remaining in the rebuttal process, we would greatly appreciate your responses to our rebuttal. If you have any followup questions or concerns, we would be delighted to address them. Otherwise, if our rebuttal has cleared up all remaining confusion, we would be grateful if you could reconsider your rating. Thank you so much!

AuthorsOpenReview ↗

Response to all

Thank you reviewers and AC for your dedication to the review process. We are excited the reviewers recognize we provide a useful and principled basis for ranking ranking states by their modeled disturbance resistance (gtQR\color{#648FFF}{\textsf{gtQR}}) and that our experimental results are very promising (BDXo\color{#E69F00}{\textsf{BDXo}}). We believe WISH takes a significant step toward constructing more performant robust safety filters with world models in latent space.


As all reviewers recognize our technical novelty, the primary criticisms stemmed from insufficient clarity on the WISH algorithm and the gap between our theoretical framework and our practical algorithm. These aspects might be unclearly stated in the initial submission, leading to unnecessary confusion and potentially misleading readers. This was never our intention. We have revised our manuscript to address the concerns raised by all reviewers. Notable changes are summarized below.

 

Clarified WISH adversarial training and implementation details

We clarified details on our adversarial training and the hyperparameters used for training and evaluation. In particular, while we did not prove guarantees for our adversarial optimization, we performed new experiments to show that our approximation works well in practice (Table 4, Table 5).

 

Clarify gap between theory and practice

Reviewer pointed out that our implementation had an additional ∣∣ϵ∣∣∞≤1|| \epsilon ||_\infty \leq 1 assumption which prevents the use of Corollary 1 for safety guarantees (assuming exact optimization). We reran our experiments without this assumption (updating Fig. 4, 5) without changes in performance, closing this theory-practice gap.

Moreover, we reiterated that we use theory as a principled foundation for algorithm design. WISH uses NN approximations and thus does not inherit the theoretical guarantees, and this is already acknowledged in the limitations section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.