acceptodds
Withdrawn from ICLR 2027

WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Assessing adversarial robustness in world-model agents requires costly, noisy closed-loop rollouts. We cast this task as a search for attack configurations against a fixed victim, subject to observation perturbation limits and a finite online budget. We present WMAttack, a closed-loop framework with two modules: Representation-Guided Attack Retrieval (RGAR) and Self-Correcting Attack Search (SCAS). RGAR matches internal and behavioral descriptors to stored attack records, retrieving configurations that give search an informed starting point. SCAS uses feedback from the current task to guide the construction, ranking, and selection of feasible candidates. An Auditor summarizes return degradation, policy-output disagreement, runtime, and return variability into structured advice. Fixed rules use this advice to build and rank candidates, and an LLM proposer selects from the resulting shortlist. A scout–confirm schedule screens candidates, confirms promising choices, and freezes the selected configuration for independent testing. Test results never feed back into search. On 26 Atari and 20 DeepMind Control tasks with DreamerV3, full WMAttack exceeds the highest-mean baseline, Successive Halving, by 0.138 and 0.133 in mean independent-test normalized return degradation, respectively, under matched online episode budgets; only WMAttack uses historical records. Controlled ablations support both modules' value, while further evaluations demonstrate applicability to three other world-model architectures.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
World Model, Adversarial Attack, Automated Attack Search

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1Ct8OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates adversarial robustness evaluation for world-model agents as a finite-budget search problem and proposes WMAttack, which combines Representation-Guided Attack Retrieval (RGAR) with Self-Correcting Attack Search (SCAS). RGAR retrieves historically effective attack configurations using representation and behavioral similarity, while SCAS uses current-task feedback and an LLM-assisted proposer to refine candidate selection under a scout–confirm evaluation protocol; controlled experiments cover 26 Atari and 20 DeepMind Control tasks with DreamerV3, with additional evaluations on other world-model architectures.

Critical strengths

The careful attempt to turn adversarial evaluation into a controlled search protocol rather than reporting only a hand-tuned attack, including scout/confirm allocation and an independent held-out testing stage.

Critical weaknesses

Matched online episodes do not imply matched attack computation: a configuration with more optimization steps or restarts can consume substantially more victim forward/backward evaluations within the same episode, and WMAttack additionally invokes retrieval and LLM components.

Questions for authors
  1. The matched-budget evaluation is a substantial improvement, but the budget is defined primarily in terms of online attack episodes. Could the authors additionally report total victim forward evaluations, backward/gradient evaluations, wall-clock time, and LLM cost for WMAttack and each search baseline?

  2. The controlled experiment uses a common discrete configuration menu and a fixed 10-pixel perturbation radius within each attack family. How does WMAttack perform when attack family, perturbation radius, optimization steps, restarts, and allocation policy are searched jointly, rather than under the more restricted controlled menu? This would better support the broader formulation of automated attack-configuration search presented in the paper.

  3. Please further analyze the sensitivity of the search outcome to the utility weights assigned to return degradation, policy disagreement, runtime, and return variance. In particular, does changing these weights alter which configuration is ultimately selected or the ranking among search methods? A Pareto-style analysis would be more informative than relying on a single scalarization.

  4. The historical ablation is somewhat intriguing because SCAS without RGAR is not uniformly beneficial in the historical evaluation, whereas the complete system performs strongly. Could the authors explain more clearly why self-correction can become ineffective or detrimental without a good retrieval initialization?

  5. The LLM-specific contribution appears statistically positive but relatively modest compared with the overall effect of the full search pipeline. Could the authors provide a stronger deterministic, LLM-free baseline using the same candidate construction, feedback features, and evaluation budget?

  6. The theoretical result decomposes finite-budget suboptimality into coverage, selection, and estimation terms, but the assumptions are fairly generic and do not directly establish why RGAR or SCAS should improve these quantities in realistic world-model evaluation. Could the authors more explicitly distinguish what is formally guaranteed from what is empirically observed?

Reviewer_QDgsOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a pipeline to identify adversarial attacks against a model that learns a world model in a closed-loop environment. The pipeline uses existing tools and frameworks from adversarial robustness (e.g., the work of Croce et al.), as well as historical data to better guide the search and produce effective attacks. The authors conduct experiments on 46 environments and two models: DreamerV3 (which learns a world model) as the target and Qwen2.5-3B-Instruct as the proposer/auditor module.

Critical strengths
  1. I appreciate that the authors try to address an important problem in the literature, i.e., that of evaluating the adversarial robustness of models beyond classic vision-language models and under different perturbation scenarios
  2. I also appreciate that the authors tried to provide a mathematical formulation of the problem; unfortunately (see weaknesses), the final result is confusing, with missing notation and undefined terminology
Critical weaknesses
  1. All the adversarial frameworks used in the paper and introduced in Section 3.2 are presented without context, which results in a list that has little meaning for any reader, even those (like myself) who are familiar with the adversarial attack literature in CV. Also, UTU_T is used for the first time at line 165 but is never introduced. I imagine that it represents the utility under a policy realization, so it corresponds to the reward function used two lines above (R(T)R_{(T)}). In particular, utility is an “unhappy” term in this context: reward is a much better choice, as in RL the utility in a given state generally assumes there is no probabilistic transition from the previous state (so, the environment is deterministic and the expected reward is entirely determined by the policy and the initial state). Also, Equation 6 is given with no context, confusing the reader and reducing the clarity of the paper
  2. My biggest concern is about the contribution and novelty of the paper: my understanding is that the paper proposes a pipeline to find adversarial attacks against DreamerV3 (with just one model, so no generalization claim can hold, a concept that is particularly important in the adversarial attacks literature [1]), but there is no real contribution other than combining existing pieces of work into a tool
  3. The term "world model" is somehow misused and never explained: DreamerV3 is not a vision-language model, but at no point do the authors address (see Questions) how they leverage its so-learnt world model to guide the attacks
  4. The bound in Section 4.4 is not formalized properly: ee is never introduced, and the resulting formula has little meaning in the context of the paper
  5. The results do not report any statistical tests, making it hard to judge whether they hold and what the variance is. Also, the authors claim, at line 361, that the same experiments have been conducted on two additional models. However, in the Appendix, they test only their suite, with no applicable baseline for comparison

[1] Hu et al., Transferable Adversarial Attacks on Black-Box Vision-Language Models

Questions for authors
  1. I understand that DreamerV3 learns a world model, but how do the authors attack a world model, besides perturbing the input pixels of an image it receives? To me, it seems that the term is a bit misused here, unless the authors can justify why perturbing some pixels with the techniques they employ constitutes a different strategy than classic CV adversarial perturbations.
  2. How is the bound in Section 4.4 used in the experiments? What is ee, and why is it not properly introduced?
  3. How is utility defined (and why is it not defined in the paper), and is it the same as the classic reward function in RL?
  4. Can the authors provide the source code to reproduce the experiments?
Reviewer_cTKTOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Evaluating the adversarial robustness of world-model agents requires numerous computationally expensive closed-loop rollouts. Therefore, the authors formulate the problem as finding an effective attack configuration under a limited computational budget. WMAttack is a closed-loop framework designed to identify a successful attack configuration from a predefined hyperparameter space under a limited rollout budget.

Critical strengths

The authors investigate the high cost of assessing adversarial robustness in world-model-based agents. This problem is particularly relevant given that such agents are being increasingly used across a wide range of tasks, which, in turn, calls for greater attention to safety.

Critical weaknesses
  • The method finds a strong configuration within a restricted set of admissible configurations. For robustness testing, it seems more informative to evaluate the agent against different attack types: an attacker without access to the agent cannot search for the best attack, but can try several popular methods.
  • The attacks assume white-box access to the agent. In practice, the attacker often has no such access, which makes the problem formulation weak.
  • Appendix J.1 describes it as a heuristic procedure. The original Claudini is an LLM agent that automatically researches attack algorithms (for jailbreaks). It is unclear why the baseline is heuristic, and its implementation is not adequately described. A strong LLM given the same information as WMAttack might not produce the results in Table 1.
  • Methods are compared by return degradation, but baselines select configurations by Utility. Utility is the authors’ own trade-off between return drop, policy-output flips, runtime, and attacked-return variance. Why should baselines select by a metric designed specifically for WMAttack?
  • According to Appendix J.2, the baselines run identically in both panels; only WMAttack loses RGAR in (b). Yet all baselines score lower in (b). No explanation is given.
Questions for authors
  • Could you provide a comparison with PriorBand (NeurIPS 2023), SMAC3 (JMLR 2022), LLAMBO (ICLR 2024), and OPRO (ICLR 2024)?
  • Does WMAttack in Table 2(a) use history before its first run? If so, where does this history come from?
  • Could configuration selection in Table 2 be done by the target metric rather than by Utility?
  • How exactly is the Claudini-style baseline in Table 1 implemented, and why is it heuristic?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.