- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies a backdoor attack against pretrained World Models. The attacker only needs to modify the World Model checkpoint before its release, without interfering with the victim's training data, reward function, or controller. Even if the victim trains a new Actor using clean data or directly uses the World Model for MPC/CEM planning, the controller may still execute an attacker-chosen action when a specific trigger appears.
The authors propose two attacks for these two control settings: Latent-Trap Attack and CEM-Plan Attack. The former modifies the latent dynamics to make a newly trained Actor prefer the target action. The latter changes the costs predicted by the World Model so that the MPC/CEM planner selects the attacker's desired action. The authors conduct experiments on DreamerV3 and LeWorldModel to evaluate attack effectiveness, different trigger types, and the impact of the backdoor on normal model performance and defense methods.
The paper addresses a meaningful security problem. As pretrained World Models are increasingly reused in downstream systems, understanding whether these models can be trusted is an important research question.
Critical strengths
S1. The research problem is meaningful, and the attack setting has some novelty.
This paper shows that an attacker can influence a victim's downstream controller by modifying only the World Model checkpoint. In particular, the backdoor can still work even when the victim trains a new Actor using completely clean data. This reveals a security risk when pretrained World Models are reused in downstream tasks, which I consider the paper's most valuable contribution.
S2. The experiments provide convincing evidence of attack effectiveness and validate the attack mechanisms.
The authors demonstrate effective attacks in both Actor and Planner settings. Through comparisons with clean models, experiments that turn the trigger on and off, and component ablations, they show that the attacks are caused by the poisoned World Model rather than ordinary visual perturbations or general performance degradation. These experiments make the paper's conclusions more convincing.
Critical weaknesses
W1. The attack may require prior knowledge of downstream tasks to select suitable target actions.
Although the authors conduct some generalization experiments, the results seem to suggest that the attack works better when the target action is consistent with the normal behavior of the downstream task. This means that attackers may need to know what actions the downstream task typically performs in order to choose an effective target action. If such detailed knowledge of downstream behavior is required, the practical applicability of the attack may be limited.
W2. The attack's dependence on the downstream controller mechanism may affect its stealthiness.
The attacker needs to know the type of downstream controller in advance and design different attacks for different control mechanisms. If the victim uses another control mechanism, the attack may fail. More importantly, since the backdoor modifies the internal dynamics of the World Model, it may still reduce normal task performance even when it fails to induce the target action. This could make the backdoor easier to detect. The paper currently lacks experiments examining this possibility.
W3. Performance drops on most normal tasks weaken the evidence for the backdoor's stealthiness.
As shown in Table 1 and Table 2, normal task performance decreases after backdoor injection in all tasks except Walker walk. Some drops are relatively large. For example, the return on Quadruped walk decreases from 697.4 to 519.1. It remains unclear whether these drops exceed the normal variation between model training runs and whether the victim could detect the backdoor through normal task evaluations.
Questions for authors
C1. Attack effectiveness under different control mechanisms
I suggest that the authors explain or provide additional experiments to examine whether the same backdoored World Model could still reduce normal task performance when the victim uses an unexpected control mechanism, even if the attack fails to induce the target action.
C2. Normal performance degradation and stealthiness
I suggest that the authors compare the performance drops in Table 1 and Table 2 with normal model performance variations, and analyze whether these drops could allow the victim to detect the backdoor through normal task evaluations.
Q1. How are target actions selected?
Does the attacker need prior knowledge of the downstream task's normal behavior to select an effective target action? How do the authors select target actions in practice?
Q2. Differences from TrojanWorld
TrojanWorld also studies how poisoning a World Model can influence the actions of downstream agents. What are the key differences between this paper and TrojanWorld in terms of their threat models and technical contributions?
2 comments
Thank you for the detailed response and additional experiments. The new results on controller transfer and clean-performance variation help address my concerns and clarify both the effectiveness and limitations of the proposed attacks.
I particularly appreciate the authors' acknowledgment that the clean-performance degradation on Quadruped exceeds normal training variation and may affect the attack's stealthiness. While this concern is not fully resolved, I believe the additional evidence and clarifications have sufficiently addressed my main questions. I am willing to raise my rating from 2 to 3.
We thank the reviewer for the thoughtful discussion and for identifying the clean-data training of a fresh downstream actor as a central contribution. We appreciate the positive assessment of the mechanism evidence and the controls separating targeted steering from ordinary visual disruption.
W1/Q1 Target actions and task knowledge. We thank the reviewer for raising this point. We chose the targets manually within the valid action space to make the induced behavior easy to recognize. For Walker, we used an alternating action pattern on both legs. These targets were not tuned separately to each downstream objective, and poisoning does not use the victim's future reward or goal.
Appendix B discusses how task compatibility can make some targets easier to enforce. However, the full Walker target occurs in only 0.21% of clean walking steps, yet the attack recovers all six joints. We also keep the same poisoned model and target when changing the victim's objective from walking to running.
In the running task, four of six joints remain controlled, while two consistently change sign. The actor therefore produces a different, repeatable action combination, while the trigger still activates the attack. Task compatibility can affect how precisely the original target is recovered, but incomplete recovery does not necessarily remove the triggered steering or task failure.
W2/C1 What happens with another controller? Following the reviewer's suggestion, we replaced the Walker actor with a CEM planner on the same supplied models. The poisoned models retain useful clean control, with mean return 853.0, but return falls to 24.2 when the trigger is added. The benign controls remain stable. Target cosine is only 0.297 for the poisoned models, so precise targeting does not transfer, but the triggered disruption remains. Changing the controller therefore leaves useful clean control alongside severe temporal trigger-gated disruption in this setting.
Walker actor-to-planner transfer (n=5 victim runs; 20 episodes/run).
W3/C2 Clean performance and training variation. We now report five downstream runs per condition to compare the clean-performance gaps with training variation. The effect depends on the task: Cheetah loses about 2% of clean return, whereas Quadruped can lose about 25%, which is larger than the observed variation between runs. We therefore cannot attribute the Quadruped drop to ordinary seed variation. A victim with a trusted performance reference could notice this loss. However, a recipient evaluating a downloaded model without such a reference would not have the same comparison.
Clean returns (n=5 downstream victim runs/group).
Q2 Difference from TrojanWorld. We thank the reviewer for pointing out this related work. Both papers study backdoors implanted through the world model. The main difference is when the downstream actor is trained. TrojanWorld uses an already trained, frozen actor or decision module as feedback during poisoning. In our actor setting, the victim trains a new actor, critic, and reward head on clean data after receiving the poisoned checkpoint. Our latent-trap construction makes this new actor learn the malicious preference through imagined rollouts. For the planner setting, we differentiate through a softCEM surrogate to influence the iterative search. TrojanWorld additionally focuses on persistence after trigger removal, while we emphasize the trigger-gated behavior as it gives the attacker more control over when the backdoor is active or hidden. We will add TrojanWorld to the related work and to the comparison in Table 18.
Ethics. We also acknowledge the reviewer's concern about releasing attack code and poisoned models. We will discuss the risks of these artifacts alongside the detection and mitigation results. To limit misuse, we could keep specific poisoned checkpoints and parts of the attack code private, making them available only upon request for reproducibility and defense research.