- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes WISH, which addresses safety filtering from images when binary failure labels are available but a meaningful continuous safety margin is not. It defines robustness as the minimum cumulative disturbance effort needed to reach failure under a fallback policy, extending LEAP to discrete-time world-model dynamics. For the exact value and admissibility condition, it proves safety under a cumulative disturbance budget and derives finite-horizon failure-probability bounds. Its implementation learns an approximate value through adversarial RL in imagination and reports improved safety-performance tradeoffs on four simulated visual-control tasks.
Critical strengths
S1. A useful adaptation of minimum-effort safety filtering to latent world models. Building on LEAP’s minimum-effort value and cumulative-budget safety principle, WISH instantiates disturbances through the world model’s noise channel and measures their effort using shifted negative log-likelihood. This provides a dynamics-dependent safety scale using binary failure labels, without requiring a hand-crafted margin or an explicit physical disturbance model. The paper develops the corresponding discrete-time filtering condition and finite-horizon failure-probability bounds under exact modeled dynamics and values. The resulting formulation provides a principled basis for ranking states by their modeled disturbance resistance, with Figure 1 illustrating the potential usefulness of this distinction.
Critical weaknesses
W1. The mechanism enforcing successful attacks is underspecified. Section 4.3 states that the adversary is penalized for failing to reach the failure set, but the explicit per-step reward is only . Appendix C.3.1 describes the reachability classifier, two value heads, and critic polishing without specifying how this penalty enters adversary optimization. Under the stated reward alone, choosing zero disturbance achieves maximal return without requiring failure. It is therefore unclear how the implemented training supports interpreting the learned value as the minimum cost of successful attacks in Eq. (8). The authors should specify the complete adversary objective, treatment of unsuccessful rollouts, and targets for both critic heads, explaining how these prevent the trivial zero-disturbance solution.
W2. The sources of the empirical gains are only partially isolated. The shared-fallback experiment in Figure 5(a) supports an advantage beyond fallback improvement on Cheetah. However, Figure 5(c) replaces the HJ margin on Waiter without clearly stating whether the fallback is held fixed. Consequently, the evidence separating improvements in the value/filter from improvements in fallback quality remains limited outside Cheetah. The authors should clarify the Waiter ablation and provide a targeted shared-fallback comparison on Walker or Waiter.
Questions for authors
Q1. What are the complete adversary reward and regression targets for both critic heads? How are unsuccessful or truncated attacks treated, and what prevents the zero-disturbance solution under the stated reward alone? Please also clarify Figure 6’s return definition and accumulation horizon: Cheetah returns reach approximately −410, whereas a 15-step sum of the stated disturbance costs cannot exceed 240.
Q2. On held-out states, how frequently do learned attacks reach failure, and how well does reflect their undiscounted disturbance costs?
Q3. Does Figure 5(c) hold the fallback fixed or retrain it with the new margin? If it is retrained, an additional shared-fallback comparison on Walker or Waiter would help establish whether the Cheetah finding extends beyond that task. Please also specify the threshold ranges used to construct the frontiers.
Q4. Under the exact-model/value assumptions, can the authors illustrate when Corollary 1 provides an informative bound while satisfying ? How does restricting the training adversary to affect the learned value’s interpretation relative to the unbounded Gaussian noise assumed in the corollary?
6 comments
Response to Reviewer BDXo (2/3)
Q2: How well does reflect the undiscounted disturbance costs on held out states?
Fairly accurately.
The learned attack reaches from 99.3–100% on held-out states, while the Spearman correlation between and the undiscounted cost ranges from 0.66 to 0.97 over tasks on held out states. On latents from the dataset, the Spearman correlation increases to 0.81 - 0.97. This indicates that there is a small generalization error on held-out states, but otherwise the learned tracks the disturbance costs well.
- Held out states: latents that were not in the training dataset
- Train states: latents from the dataset used to train the world model, adversary and values.
| Task | Held-out states | Train states |
|---|---|---|
| Cartpole | 0.66 | 0.88 |
| Cheetah | 0.68 | 0.81 |
| Walker | 0.84 | 0.82 |
| Waiter | 0.97 | 0.97 |
We have added this discussion and table to Appendix C.3.5 in the revised pdf.
W2, Q3: In Figure 5c, does use the same fallback as WISH?
No, doesn't use the same fallback as WISH. In Figure 5c, retrains its own fallback policy against the new margin , so that comparison changes the margin and the fallback together.
As suggested, we ran a shared-fallback comparison on Waiter and added it to Figure 5c in the revised pdf as , which is but with the fallback policy initialized and frozen to WISH's fallback .
We make the following observations:
- still improves over , confirming that the WISH fallback is a large part of the gain on Waiter.
- The remaining gap to WISH at matched performance shows that the minimum-effort value contributes beyond the fallback, consistent with the Cheetah result in Figure 5a.
What about ?
One can also consider an ablation where we use WISH's value as the margin and fix the fallback to be WISH's fallback to try and "isolate" the HJ value backup versus how WISH learns the value. However, one can show that the WISH value is the solution to the HJ backup when using WISH's as the margin and WISH's fallback (Appendix B.5). Thus, this variant does not "isolate" HJ value versus WISH value since the values coincide theoretically. We were able to confirm this empirically and found that the fronts basically coincided (Figure 8 in the revised pdf).
Q3: Please also specify the threshold ranges used to construct the frontiers.
For each method, we use 20 linearly spaced thresholds in , crossed with . Ceiling is the 99th percentile of that method's deployed value over safe latents from the world-model dataset, averaged over the three training seeds. For MPS, is the 95th percentile of its margin, and there is no . The resulting ranges are:
Main results (Fig. 4)
| Task | WISH ( on ) | HJ (GP) ( on ) | HJ (no GP) ( on ) | MPS (margin ) |
|---|---|---|---|---|
| Cartpole | ||||
| Cheetah | ||||
| Walker | ||||
| Waiter |
Ablations (Fig. 5)
| Panel | WISH | WISH (HJ fallback) | HJ (GP) | HJ + | HJ + WISH |
|---|---|---|---|---|---|
| a. Cheetah | – | – | |||
| c. Waiter | – |
Dash means not applicable. Note that Fig. 5b has no threshold since it runs each fallback alone, without a filter. In addition, recall that WISH ranges are in units of disturbance cost and so differ across tasks; the HJ values are bounded by 1 (tanh), so their ranges are close to . We have added these tables and the rule above to Appendix C.3.4 (Tables 2 and 3) in the revised pdf.
Response to Reviewer BDXo (3/3)
Q4: Under the exact-model/value assumptions, can the authors illustrate when Corollary 1 provides an informative bound while satisfying ?
Good suggestion. Our existing toy example provides a concrete illustration. Here, . Choosing and , Corollary 1 gives
\leq 1-F_{\chi^2}(25;10) \approx 0.005346,$$ certifying a safety probability of at least **99.465% over N=10 steps**. **We add additional discussion on the informativeness of the bound from Corollary 1 in Appendix B.11.1**, and have added the following paragraph to the main text: > The informativeness of Cor. 1 depends mainly on the event inclusion in Thm. 1. For fixed D, the upper bound can become conservative over longer horizons N. We illustrate this on the toy example (App. B.11) in App. B.11.1, showing both an informative certificate over short horizons and this conservatism over longer horizons. ## **Q4:** How does restricting the training adversary to $|| \epsilon ||_\infty \leq 1$ affect the learned value's interpretation relative to the unbounded Gaussian noise assumed in the corollary? Good catch! Indeed, from the perspective of Corollary 1, it is more accurate to use an unbounded action space for the adversary. The current $||\epsilon||_\infty \leq 1$ constraint can be interpreted as changing Corollary 1 to consider truncated Gaussian noise instead of standard Gaussian noise. To assess the effect of this implementation choice, we **rerun our experiments with an unrestricted adversary**. We find that **performance is essentially unchanged on all tasks** except for Waiter, where the method drops in performance relative to the $||\epsilon||_\infty \leq 1$ variant but still outperforms all the baselines (Figure 7 in the revised pdf, will replace Figure 4 after the updated ablations finish running). We therefore adopt the unrestricted implementation for the main results in the revised pdf. We are not yet able to explain why the $||\epsilon||_\infty \leq 1$ performs better only on Waiter. We leave this to future work.
Response to Reviewer BDXo (1/3)
We thank the reviewer for raising valuable comments. We are excited that the reviewer identified that WISH provides "a useful adaptation of minimum-effort safety filtering to latent world models" and defines a "principled basis for ranking states by their modeled disturbance resistance". We believe that WISH takes a significant step toward constructing better robust safety filters in latent space. We try our best to resolve all raised concerns in the responses below.
W1, Q1: Can you clarify how the adversary's optimization problem is solved?
In the updated version of the manuscript, we have refined our explanation on how the adversary's optimization problem is solved to enhance readability. Specifically, we have elaborated on the mechanism that guides successful attacks:
What happens to unsuccessful attacks? Attacks that are unsuccessful at the end of the rollout (and thus truncated) bootstrap their value and accumulate potentially infinite costs, in comparison to successful attacks that terminate the episode and have finite cost. This incentivizes the adversary to perform successful attacks. However, discounting means that an unsuccessful attack could have a lower cost than a successful attack. We already acknowledge this in our Limitations section.
Despite this approximation, in practice the learned attacks nearly always reach failure (see answer to Q2).
Q1: Please also clarify Figure 6's return definition and accumulation horizon.
Thank you for catching this. We were previously plotting the mean of the value target which includes a bootstrap and thus corresponds to the cost of the whole attack until failure rather than the cost of the 15-step window. We have corrected Figure 6 in the revised pdf to directly the following directly measured quantities:
Q2: On held-out states, how frequently do learned attacks reach failure?
Nearly always. The adversary reaches failure in over 99% of the held-out states over all tasks.
For each task we take 400 held-out states (i.e., from episodes the world model never trained on) for each of the 3 WISH seeds, and roll out the learned adversary against the fallback policy for 300 steps in imagination.
We have added this discussion and table to Appendix C.3.5 in the revised pdf.