acceptodds
Under review as a conference paper at ICLR 2027

Toward Adversarially Robust World Models

Abstract

Latent dynamic world models have proven to be highly versatile tools for embodied artificial intelligence systems, enabling online action planning, safe policy evaluation, and scalable policy learning. However, this versatility makes world models a vulnerable and centralized point of failure as a compromised model can impact planning, evaluation, and training at once. Therefore it is critical that the security and robustness properties of world models are properly understood before they are universally deployed in embodied AI systems. In this work we take a unifying approach towards this understanding, developing two novel attacks against world models, named state hallucination and dynamic shift attacks, aimed directly at impacting downstream systems, and proposing a new theoretical lens through which to study the robustness of latent dynamic world models. Through our analysis we identify robust latent space encoders as the key component of robust world models. We empirically validate this finding on four separate latent dynamic world model families, studying Dino World Model, Le World Model, r2Dreamer, and Cosmos 3, demonstrating significant improvements in robustness through adversarial encoder model training.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, Security, Robustness, Embodied Intelligence, Robotics

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_oAqmOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the robustness of latent world models against perturbations on observations under the white-box setting. It proposes two types of attacks, including state hallucination which forces the model to initiate its rollouts from an erroneous state, and dynamic shift which tampers with the model's transition dynamics. It also introduces TA-MDP framework to formalize the robustness of world models, proving that while a robust predictor alone is insufficient to guarantee robustness, a robust encoder is sufficient. Based on these theorems, it employs adversarial training using a method called FARE to fine-tune the encoder on task-specific data. Experimental results demonstrate that proposed attacks severely degrade the performance of world models, as well as downstream planning and policy, and FARE significantly mitigates these adverse effects.

Critical strengths
  1. This paper focuses on an important but underexplored problem that world models serve planning, policy evaluation, and policy learning in embodied AI, and a compromised world model can affect multiple downstream systems, so their robustness is well worth studying. And it also presents a complete study including two types of attacks considering state and transition, and TA-MDP framework and extended FARE for encoder-based adversarial training.
  2. This paper conducts comprehensive experiments covering four diverse world models, evaluating the attack performance on both the world models and downstream systems such as policy eval, action planning, and policy learning, as well as the effectiveness of the FARE defense. It further provides numerous comparisons and ablations and also reports inconclusive findings.
Critical weaknesses
  1. Table 1 notes ‘scale is environment-specific’, but lacks further elaboration. It could lead to confusion when making horizontal comparisons of similar conditions within the table, for example, 'Dino WM appears most vulnerable in the Push-T task, whereas Le WM shows greater weakness in the Wall task' (line 348). It would be helpful to provide a brief clarification regarding the units themselves or the quantities being compared here, although some illustrations are shown in Appendix.
Questions for authors

1.Can FARE be compared with other defense methods, that is, other than adversarial training, to further prove its effect and priority?

2 comments
AuthorsOpenReview ↗

Response to Reviewer oAqm

Thank you so much for your positive assessment of our paper, we greatly appreciate it. In the following response we aim to answer your questions and hopefully increase your confidence in our results. If you have any additional questions please feel free to ask us, otherwise please see our responses to the other reviewers for further details on how we improved our paper.

W1 “Table 1 notes ‘scale is environment-specific’, but lacks further elaboration.”

This is a fair critique. Therefore we’ve standardized all of our reported metrics so they’re all on the same scale. We’ve also added some additional interpretable metrics which measure distances directly in latent space rather than over state probes. Our updated metrics, HrH_r and DrD_r, are all normalized with respect to the distance between the true and adversarial target latents, and are therefore all on the same scale as well. We have found these latent space metrics to be more reliable, since the linear probes themselves often come with a high level of inaccuracy even given clean inputs (see our updated Figure 4 results on r2Dreamer).

Q1 “Can FARE be compared with other defense methods, that is, other than adversarial training”

This is a fair point, therefore in Table 11 and 12 we include additional evaluation against the JPEG compression defense [1] when targeted by a BPDA style “Expectation Over Transformation” attack [2]. The results show that the defense is able to mitigate the attack’s effect to an extent, but is unable to reach the complete nullification of FARE against this attack across Dino, Le WM, and r2Dreamer on Wall, Push-T, and Reacher respectively. The defense’s effectiveness also seems to be environment or world model dependant, as it works well on Le WM Push-T, but fares much worse on Dino Wall and r2Dreamer Reacher. The defense also fails to prevent the impact of state hallucination from reaching planning, as success rates drop from 86% -> 24% on Le WM Push-T and episodic returns in r2Dreamer reacher drop from 805 -> 91. It is also possible that more complicated defense techniques, such as certified robustness, can be adapted to world models, however such adaptation would require its own analysis and paper to be properly explored and implemented.

[1] Guo et. al. “Countering Adversarial Images Using Input Transformations”

[2] Athalye et. al. “Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples”

Thanks for the authors' response. It is helpful to have such a clearer illustration of the definition and a more comprehensive experiment on defence performance.

Reviewer_5zC6OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies adversarial observation perturbations against latent world models. It introduces state-hallucination and dynamic-shift attacks, proposes a tessellated adversarial MDP formalism, and adapts FARE-style encoder fine-tuning as a defense. Experiments span four world-model families and connect representation-level manipulation to downstream planning and policy evaluation.

Critical strengths
  1. The evaluation connects latent-space attacks to consequential system behavior. Closed-loop planning failures and reversals in policy-evaluation rankings demonstrate effects beyond embedding displacement, while encoder fine-tuning recovers meaningful downstream performance.
  2. The defense evaluation includes adaptive attacks, stronger optimization, transfer tests, and unseen-scene evaluation for Cosmos. Architectural analyses also explain differences in susceptibility, particularly r2Dreamer’s resistance to dynamic shifts, providing insight beyond aggregate attack success.
Critical weaknesses
  1. How fully does the state-preserving dynamic-shift experiment separate altered dynamics from altered perceived state? Table 12 constrains and evaluates agent position, whereas the Push-T state also includes block position and orientation. Preserving agent position alone does not establish that the remaining task-relevant state is unchanged. Although the action-space analyses support systematic remapping, the distinction from state hallucination is incomplete without accounting for these other state components.
  2. What physical changes underlie Cosmos’s latent-space state-hallucination successes? In Table 24, the base model reaches 70% latent success but only 15% targeted end-effector-position success at ten denoising steps; the corresponding rates at thirty steps are 57% and 9%. Thus, proximity to a target rollout in latent space does not consistently establish hallucination of the targeted physical state. This limits the interpretation of state-hallucination success across the four model families.
Questions for authors
  1. Which encoder property is central to the robustness principle: distinguishability between observation cells, invariance within cells, or both?
  2. Does Theorem 1 require nonconstant transitions across latent cells rather than merely a nonconstant predictor? A predictor (P(z,a)=g(a)) can be nonconstant while remaining unaffected by any observation perturbation.
  3. Under the state-preserving dynamic-shift attack, are block position and orientation preserved alongside agent position?
  4. Which physical state changes explain the Cosmos attacks that satisfy the latent criterion but not the targeted end-effector criterion?
  5. What determines whether latent-only encoder distillation suffices or predictor-aware training signals are needed, as in the final Cosmos configuration?
  6. What explains the remaining downstream failure under dynamic shifts—for example, Le WM’s 18.8% defended execution success versus 81.2% clean success at the training budget?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 5zC6

Thank you for your thoughtful review and insightful questions. Below we provide further clarification for your questions along with additional experimental results when applicable. Please let us know if there are any other questions we can answer to improve your confidence in or assessment of our results.

W1/Q3 State preserving dynamic shift

We added some additional experimental results into the appendix in Table 9 where we measure the state of the T shaped block after dynamic shift attacks. In short, when using the state preserving approach we’re able to reduce the block error from 18,7 state units down to the clean input floor (probe error given a clean input) which is 11.8 state units. Here state units represent a single pixel in the environment’s 512x512 images, so a 11.8 state unit shift in one direction would only cover 0.2% of the playable area. Similarly we reduce the block’s angular error from 10.2 degrees to 6.8 degrees, which is very close to the probe’s base error of 6.1 degrees on clean inputs.

W2/Q4 “Which physical state changes explain the Cosmos attacks that satisfy the latent criterion but not the targeted end-effector criterion?”

We believe the main reason for this discrepancy is down to two key factors 1. It is harder for the probes to predict end effector positions in Cosmos since the camera position/pose isn’t fixed. 2. The attacks scramble the environment latents a bit, making this prediction even harder. (see our updated results using latent space metrics). As a result we’ve added additional latent space metrics to our Table 1 results for improved interpretability.

Q1 “Which encoder property is central to the robustness principle: distinguishability between observation cells, invariance within cells, or both?”

Both properties are certainly important for robustness but for different reasons. If an encoder perfectly distinguishes between observation cells as in Definition 2 (i.e. no observations from different observation cells map to the same latent), then one can construct a predictor such that the resulting world model is robust. In this case the predictor is the one that needs to be invariant within latent cells.

Your invariance within observation cells case is exactly what we consider in Theorem 2. Such an encoder maps all observations within the same cell to the same latent, so using it with any predictor necessarily results in robustness. Such an encoder is useful as a theoretical tool and optimization target, as we do with FARE, though actually achieving such perfect robustness isn’t really feasible in practice. FARE aims to solve this by setting distinct clean encoding as the optimization target for each latent cell, so the approach ultimately optimizes both invariance and distinguishability.

Q2 “Does Theorem 1 require nonconstant transitions across latent cells rather than merely a nonconstant predictor?”

Yes you’re right that the key condition for Theorem 1 was to be non-constant over latent cells rather than just non-constant in general. The reason for this “non-constant” condition was to allow us to distinguish trivially robust predictors (those that just predict the same value/latent cell) from those that are useful in practice. For our revision, and in line with the advice of reviewer dDwn, we ultimately ended up reworking our formulation of Theorem 1 slightly. In short, we now properly define “clean accuracy” and “robust accuracy”, enabling us to make this same distinction between trivially robust models and useful models without relying on the “non-constant” condition. The ultimate finding remains, robust predictors are insufficient, but now the result is even stronger and more clear.

Q5 “What determines whether latent-only encoder distillation suffices or predictor-aware training signals are needed,”

Cosmos is by far the most complex of the four models we evaluated and, in particular, its predictor conditions on a number of extra signals aside from pure latent encodings and actions (text prompts, text descriptions of the scene, etc.). Therefore we believe that by passing gradients through the predictor both the attacks and FARE are able to properly align their updates with this extra conditional information.

Q6 “What explains the remaining downstream failure under dynamic shifts”

We believe that the FARE encoder we presented in the original draft wasn’t trained sufficiently long enough, therefore our updated results include a FARE model trained on more data. We found that this model is much more effective against dynamic shift attacks, maintaining a planning success rate ~80% on both Le WM tasks (Figure 2). Note that we reran our closed loop planning experiments with a higher sampling size, as requested by reviewer dDwn, and fixed some bugs in our code, so exact values have changed.

Thank you for the clarifications and additional state-preservation results.

For Cosmos, additional latent metrics do not explain what physical changes occur when the latent criterion succeeds but the end-effector criterion fails. How should these cases be interpreted?

Regarding the updated planning results, which bugs affected the reported results, and were both baseline and defended models re-evaluated using the same corrected pipeline and attack settings?

AuthorsOpenReview ↗

Response to Further Questions

Hello, thanks for the additional questions and discussion.

1. Your first question actually lead to some further investigation and findings that you might find interesting. Recall that the end effector positions reported in Table 25 are approximated using a CNN based state probe. This probe is trained on all three camera views used and predicted by Cosmos 3 on Droid, one wrist camera and two external cameras. It is fairly well known that external cameras are much more important for localizing the exact position of a robot arm [1], therefore we can expect that the CNN predictor relies more heavily on the external camera views.

One issue for the attack, however, is that the wrist camera view changes significantly between the clean and target frames (since the arm moved, the whole camera view is changed), while only the arm moves in the external cameras (the remainder of the scene stays fixed). Therefore our attack, which weighed all camera views equally, effectively under valued the importance of external camera views. As a result many attack rollouts given the base predictor show the wrist camera view changing significantly while the external cameras remain unchanged from the clean trajectory. Since the state predictor relies on these external cameras it predicts states closer to the true, clean state (resulting in EE attack success failure). However, since the wrist camera view changes significantly and gets closer to the target latent, the latent attack success condition is met. We've added an additional figure and discussion on this topic to the end of the Appendix for more details.

This outcome seems to point towards a stronger approach that focuses the attack on the arm position in the external camera views, perhaps with some masking, which is something we can perhaps try for a camera ready version of the paper.

[1] Lee et. al. "Camera-to-Robot Pose Estimation from a Single Image"

2. The bug fixes were essentially only for Le WM and Dino WM. In short we found cases where the wrong logs were being read for both models when computing planning success rates, resulting in some successful plans being labeled as failed. Additionally we found that the dataset provided for Dino WM had misaligned state labels on the Wall environment resulting in all Wall results having a higher linear probe error. We reran all impacted experiments for both the FARE and base encoders to ensure fairness. This is why planning performance increased for Dino and Le WM from the original draft under both encoders. As a side note, in the original submission we found that some of the test samples used to evaluate FARE were also in its training set. For the updated draft we fixed this, ensuring a complete train test split, though the results weren't meaningfully impacted.

Thank you again for engaging with us in further discussion, please let us know if you have other questions.

Thanks for the reply. The current score 3 is still appropriate. I'll keep my score.

Reviewer_dDwnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes two attacks on latent world models: state hallucination, which makes the model misidentify the current state, and dynamic shift, which changes the predicted consequences of actions. It also introduces TA-MDPs, which partition observations into perturbation cells to define robustness, and adapts the existing FARE adversarial encoder fine-tuning method to world models. Experiments on Dino WM, Le WM, r2Dreamer, and Cosmos 3 show that the attacks affect planning and policy evaluation, and that encoder fine-tuning substantially mitigates them on several tasks. Attacks on policy training remain inconclusive. (§3–4, Appendix D.3)

Critical strengths
  1. The attacks have clear implications for decision-making. I find dynamic shift particularly interesting because it targets the correspondence between actions and their predicted consequences, beyond increasing prediction error. The appendix examines action features and uses interventions to test their role. The attack still changes action directions with a state-preservation constraint. I find this stronger evidence for the mechanism than failure videos alone. (§3, Appendices D.1–D.2, Tables 6–7 and 12)
  2. I appreciate the breadth of experiments and the controls. The paper covers planning, policy evaluation, and several world model architectures. Table 20 tests combinations of robust encoder and predictor training, Table 25 tests unseen DROID scenes, and Table 26 evaluates stronger and transferred attacks. These results give me more confidence in the defense benefits and in prioritizing the encoder as a component to protect. (Tables 2–3, Appendices D.4–D.5)
Critical weaknesses
  1. The current proof does not establish Theorem 1. Line 727 of Appendix B asserts P∗(E(xk),a)∉img⁡(P∗(E(Cxk),a))P^*(E(x_k),a)\notin\operatorname{img}(P^*(E(C_{x_k}),a)) while also stating that xk∈Cxkx_k\in C_{x_k}. By the definition of a set's image, the former must belong to the latter. Moreover, under a fixed latent partition, a non-constant predictor need not produce outputs in different cells, so lines 718–719 are also insufficient for the subsequent argument. I would like the authors to clarify how the partition is fixed, how it is handled when the encoder changes, and what additional conditions ensure distinct output cells. This affects my assessment of the theoretical contribution, but I do not take it as invalidating the attack experiments. (§3.1, Appendix B)
  2. Inconsistent reporting makes key closed-loop results difficult to verify. Table 3 states that results use 50 planning runs, whereas Table 4 lists 20 for Dino WM and 32 for Le WM. Table 14 states that the relevant rows use 20 episodes, yet reports success rates such as 0.86 and 0.96 without explaining whether they are averaged across seeds. More directly, lines 1154–1156 of Appendix D.3 report defended Le WM success rates of 78.1/75.0/53.1/18.8%, while the corresponding values in Table 15 are 81.2/68.8/43.8/12.5%. I recognize that these differences may come from separate experimental batches or text that was not updated, but I cannot tell from the current description. Since closed-loop performance is central to the paper's evidence, I would need consistent settings, raw success counts, and uncertainty estimates; correcting one number would not resolve the broader ambiguity. (Tables 3–4 and 14–15)
Questions for authors
  1. Theoretical clarification: Can you correct the set relation in line 727 of the proof of Theorem 1 and state the additional conditions needed to infer distinct output cells from a non-constant predictor? Does the latent partition remain fixed when the encoder changes?
  2. Results verification: Do Tables 3, 4, 14, and 15 come from different experimental batches? Please specify the episode count, number of seeds, and averaging procedure for each result, and explain the numerical differences between the text of Appendix D.3 and Table 15.
3 comments
AuthorsOpenReview ↗

Response to Reviewer dDwn

Thank you so much for your positive review of our paper, we greatly appreciate it. We also want to thank you for your close reading of our paper and theory. We believe our theory and empirical results have both improved significantly as a result of your suggestions. If you have any other clarifying questions please let us know, we look forward to further discussion.

W1/Q1 “The current proof does not establish Theorem 1”

Thank you for your careful reading of our theory, we really appreciate this feedback. We noticed similar issues shortly after submission and have therefore worked towards improving our theory in the interim. In short we we’ve made the following fixes to our theory that directly fix the issues you raised:

  1. In the main body we defined latent cells as a Voronoi tessellation centered at E(x)\mathcal{E}(x) while our proofs used encoder images. These two are not equivalent as you pointed out and, crucially, the encoder images used in the proof of Theorem 1 make the logic incorrect. Therefore in our updated draft we commit to a latent cell formulation. Formally, for each canonical observation xx, let CxEC_x^{\mathcal{E}} denote the latent cell of Z\mathcal{Z} centered at E(x)\mathcal{E}(x), with ties resolved by the same rule as in Eq. (5). By indexing cells by xx we can keep all latent cells disjoint even if their encodings collide (in case of collision, one of them will have an empty latent cell). Definition 2 is unchanged apart from writing these cells in place of CzC_z.

  2. We formulated our original proof by fixing P\mathcal{P} first and then quantifying over E\mathcal{E} which doesn’t work, as you pointed out, since the predictor depends on the encoder. Therefore in our updated theory we fix the encoder first and then quantify over the predictor. As a result changing the encoder results in evaluating a wholly new world model with a predictor properly defined over the latent space of its respective encoder.

  3. Our “non-constant” condition for the predictor wasn’t the right choice as you pointed out. Therefore we reframed the proof as you suggested: the predictor must send the clean encodings of two colliding observations to different latent cells under some observation. I.e. P\mathcal{P} sends encodings xx and x’x’ to different latents under some action aa, but there are some x~\tilde x and x~’\tilde x’ in the cells of xx and x’x’ respectively that get mapped to the same encoding by E\mathcal{E}. Under these conditions the resulting world model cannot be robust given E\mathcal{E}.

This last fix in particular ties in well to clean accuracy, which our original formulation ignored. A predictor which is accurate on clean data will map two distinct E(x)\mathcal{E}(x) and E(x’)\mathcal{E}(x’) to distinct next latents, given some action aa, if the real environment also makes different transitions in each observations’ corresponding states. Therefore we propose some additional tweaks to further strengthen our theory and tie it to existing formulations of clean and robust accuracy [1]. In short we define the following accuracy properties of world models :

clean accurate if P(E(x),a)∈CT(x,a)E\mathcal{P}(\mathcal{E}(x),a)\in C^{\mathcal{E}}_{T(x,a)} for all x∈Xx\in\mathcal{X} and a∈Aa\in\mathcal{A} i.e. the world model transitions match the real environment’s transitions.

robustly accurate if P(E(x~),a)∈CT(c(x~),a)E\mathcal{P}(\mathcal{E}(\tilde x),a)\in C^{\mathcal{E}}_{T(c(\tilde x),a)} for all x~∈X\tilde x\in\mathbb{X} and a∈Aa\in\mathcal{A}, i.e. every perturbation of xx yields the true next state of xx, the model is both robust and accurate.

Note that here we are somewhat abusing notation here, using T(x,a)T(x,a) as a shorthand for T(ϕ∗(x),a)T(\phi^*(x),a). With these new properties we define our final version of Theorem 1 as follows:

Let E\mathcal{E} be an encoder that is not robust. Therefore there exist some x≠x′∈Xx\neq x'\in\mathcal{X}, such that x~∈Cx\tilde x\in C_x and x~′∈Cx′\tilde x'\in C_{x'} but E(x~)=E(x~′)\mathcal{E}(\tilde x)=\mathcal{E}(\tilde x'). If T(x,a)≠T(x′,a)T(x,a)\neq T(x',a) for some a∈Aa\in\mathcal{A}, then no predictor P\mathcal{P} makes (E,P)(\mathcal{E},\mathcal{P}) robustly accurate.

Equivalently, any world model that is accurate on clean data cannot be robust if E\mathcal{E} is not robust. The updated proof in the Appendix follows the exact reasoning outlined above and as recommended by the reviewer, but uses the language of robust and clean accuracy to give the results more meaning. We believe that this updated formulation and proof greatly strengthens the results of our paper, however if the reviewer finds the changes too significant we are happy to remove the clean accuracy and robust accuracy additions in favor of a more minimal fix to the issues raised.

[1] Zhang et al., “Theoretically Principled Trade-off between Robustness and Accuracy,” ICML 2019.

AuthorsOpenReview ↗

Response to Reviewer dDwn

We apologize for the long response split between two comments. We wanted to be sure that we properly addressed your concerns with Theorem 1. We now address your other concern with results reporting.

W2/Q2 “Inconsistent reporting makes key closed-loop results difficult to verify”

Thank you for bringing this up, you are correct that the results came from different batches and we forgot to update the values. This is largely a consequence of coordinating experiments across multiple world model repos and experiment types, we apologize for the mistake. To fix this inconsistency and improve confidence in our results we reran all of our planning experiments under a unified implementation (sampling, averaging, etc.) with a higher number of samples/seeds for many experiments (50 for closed loop results). These new results don’t change our core findings but help certify our original claims. The updated manuscript includes these new results and ensures that the appendix and main body report from the same experiment batches.

Given the increased page limit we also converted some tables to figures for improved readability and also included some additional experiments and metrics requested by the other reviewers. Note that while these changes are visually substantive, they don’t change the underlying conclusions of the paper, instead giving more evidence in support of our claims.

AuthorsOpenReview ↗

Quick Follow Up

Hello Reviewer dDwn,

Thank you again for your time and for your helpful review. We're curious to hear your opinions on our updated theory and results. Do you think the updated theory addresses your concerns? Does our new formulation and more correct proof change your assessment of the paper? It would be great to know if you have any additional lingering questions or feedback.

Best,

Authors of Submission #49158

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We first want to thank the reviewers for their positive assessments of our paper along with their insightful feedback. We’re glad to hear that everyone appreciated the framing of our paper and viewed its results as important for the field. Our goal in this rebuttal revision is therefore to strengthen and improve confidence in our results. To this end we’ve made the following updates to the paper:

  1. Reviewer dDwn pointed out a flaw in our proof of Theorem 1 and reviewer 5zC6 raised some questions around our usage of “non-constant” predictors as a condition in our proof. Therefore we’ve improved the rigor of our theoretical results by first directly fixing the issues raised by reviewer dDwn with a revised proof and a more consistent theoretical formulation. We further drop the “non-constant” predictors condition, instead conditioning on the clean accuracy of the predictor. Overall these fixes, as recommended by the reviewers, make our results both stronger and more clear.

  2. Reviewer dDwn raised concerns about inconsistencies in our results reporting. This issue arose largely as a result of coordinating experiments across multiple world model codebases. We’ve therefore rerun these experiments along with a number of others using an increased sample count to ensure consistency and accuracy. Also note that we fixed some bugs in our code which caused lower probe and planning scores. These fixes improved results for both the base and FARE encoder. If there are slight changes in numbers reported between the original and revised submissions, this is why.

  3. Reviewers 5sC6 and oAqm noted some oddities in the behavior and interpretability of our linear probe metrics. We’ve therefore added additional latent space metrics to our results reporting and analysis. We believe that these metrics are more interpretable and better align with our goal of measuring impacts on planning, training, and eval as all three systems consume latents not linear probe readouts. See figures 6 and 7 to see the new metrics plotted visually. For consistency all linear probe metrics are still reported in the paper. Linear probe scores are now standardized as recommended by reviewer oAqm.

  4. Reviewer oAqm noted that we only compared against adversarial training based defenses. Therefore in Appendix Tables 11 and 12 we compare against a JPEG compression defense and show that its attack mitigation effects are significantly weaker than that of robust encoder training and much more environment dependent.

Given the extra page afforded during the rebuttal, we’ve added a number of additional experimental results to the main body of the paper. These include:

  • Figure 2 has further planning results for Le WM and Dino WM against dynamic shift and untargeted attacks along with policy rollout results against r2Dreamer (aimed at being analogous to planning). Previously we only reported state hallucination attacks vs Dino and Le WM.

  • Figure 3 has a more thorough evaluation of adversarially trained model predictors (on Dino and Le WM), giving further support of our Theorem 1 claim.

  • Figure 4 has further PCA and robust probe analysis across all models and environments (excluding Cosmos 3).

We believe that these updates greatly improve the strength of our claims and hopefully improve the reviewers’ confidence in our paper. We are excited to hear back from the reviewers based upon these updates and openly welcome any additional questions or critique.

Thank you, Authors of submission #49158

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.