- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper identifies a new attack vector targeting LLM agents that rely on world models to predict the outcomes of their actions. The authors argue that these world models are likely to be outsourced due to how expensive these are, creating an attack surface through which poisoned world models can steer agents toward attacker-desired behavior. They propose a contrastive attack method to exploit this vulnerability and show that it increases the attack success rate (ASR) while largely preserving utility on benign tasks.
Critical strengths
None
Critical weaknesses
-
The practical motivation for using an external world model is insufficiently established. It remains unclear when this setup offers meaningful benefits over an agent interacting directly with the environment.
-
The attack appears to rely on restrictive assumptions about agent behavior. In the provided example, the agent seems to follow the world model’s predictions without verifying them against environmental feedback, raising questions about the attack’s applicability to agents that update their plans based on actual observations.
-
The methodological novelty is unclear. The approach appears to rely primarily on modifying data labels, and its distinction from existing data-poisoning methods is not sufficiently explained.
Questions for authors
-
Could the authors provide concrete examples or experimental evidence demonstrating the benefits of using an external world model?
-
Does the attack remain effective when the agent incorporates feedback from the actual environment and revises its plans accordingly?
-
How does the proposed contrastive attack differ from existing label-manipulation or data-poisoning methods? Is the primary contribution the identification of a new attack surface or the attack technique itself?
We thank the reviewer for the time and the questions, and we appreciate the opportunity to clarify. We answer the three points in turn and, for each, also point to where the paper discusses it. The revised PDF, uploaded during the discussion period, makes these places easier to find.
W1 / Q1: Why plan with a world model, and why an external one?
Why a world model. A WM does not replace interaction with the environment. As described in Sec. 3, the agent still takes an observation and executes an action in the real environment at every step. The WM lets it predict the outcome of each candidate action before committing to one. This matters when trying an action has a cost or cannot be undone, such as placing an order or a physical action; the first paragraph of Sec. 1 names these benefits as "improving task success, reducing execution costs, and avoiding irreversible consequences". Planning with WMs is an active line of work, summarized in Sec. 2 ("Agent planning"). WebDreamer (Gu et al., 2025) and WMA (Chae et al., 2025) use WMs to choose the actions of web agents, RAP (Hao et al., 2023) uses one for reasoning, and Li et al. (2026) train text WMs on agent trajectories and use them to verify irreversible actions before execution. These works report the benefits of WM-based planning.
Why external. "External" refers to who trains the WM, not to how the agent interacts with the environment. As noted in Sec. 1 and at the beginning of Sec. 4, an accurate WM needs large trajectory data from the environment, compute, and domain expertise; the training datasets provided by Li et al. (2026), which our experiments use, contain 40,000 to 70,000 trajectories per environment (Sec. 6.1). A developer without these resources takes a trained WM from a third party or outsources its training. This is the supply-chain setting of outsourced training in backdoor attacks (Gu et al., 2017). Our paper studies what such an untrusted WM can do to the agent that uses it.
W2 / Q2: Does the attack hold when the agent uses environment feedback?
Yes. The agent in all our experiments already does this. At every step it proposes candidates from the current real observation (Sec. 3, Eq. 1), and, as stated in App. D.3, "Only the executed action and its real observation are appended to history." The WM's predictions only score the candidates of that step and are then discarded. The agent therefore revises its plan from real feedback at every step. Figure 1 illustrates one planning step of this loop.
Feedback cannot undo the attack, because the attack succeeds when the target action is executed. The real observation arrives after the action: the agent has already opened or bought the promoted product, or the measurement procedure has already been broken. The 43.6% to 70.0% ASR@20 that CONMAN reaches in Table 1 is measured with this feedback loop. The revised PDF states the feedback loop explicitly in Sec. 3.
W3 / Q3: How does CONMAN differ from data poisoning, and what is the contribution?
The attack goal differs from conventional data poisoning. Conventional poisoning changes the behavior of the poisoned model itself, by writing the desired output into that model's training data; in the words of Sec. 2, such attacks "directly embed the target output, e.g., an action or answer, into content consumed by the attacked model or agent". Here the poisoned model is not the one that acts. The attacker wants a separate agent to change its behavior, without access to the agent's components, inputs, or data (Sec. 4). App. A (Table 4) compares CONMAN with seven recent attacks on WMs in this respect. This setting raises two challenges, discussed in Sec. 5.1, and CONMAN addresses each.
The attacker cannot write the desired action anywhere. It can only change what the WM predicts, and an evaluator it does not control turns those predictions into scores. CONMAN therefore writes next-state labels that make the target action look like progress, and selects them with a surrogate evaluator.
Raising the target is not enough, because competing actions at the same state can still score higher. CONMAN identifies these competing actions and demotes them, so that the target becomes the best-scoring candidate.
Conventional label poisoning applied to the WM is our Promotion-only baseline (Sec. 6.1). At the same 2% poisoning budget it reaches 20.0% to 27.5% ASR@20, against 43.6% to 70.0% for CONMAN (revised Table 1). The contribution is both, as listed at the end of Sec. 1: the attack surface, in which a poisoned WM steers a separate agent, and the method that makes the attack work under its two challenges.