acceptodds
Under review as a conference paper at ICLR 2027

BadWAM: When World Action Models Dream Right but Act Wrong

Abstract

World-action models (WAMs) are emerging as a promising foundation for embodied control: rather than predicting actions alone, they learn representations that couple action generation with future world prediction. This coupling is often viewed as a source of robustness, interpretability, and safety, as a robot's action can in principle be checked against its imagined future. In this paper, we show that this assumption is fragile. We introduce **BadWAM**, a unified framework for modeling **World-Action Drift Attacks**: a new class of WAM-specific adversarial attacks that use small visual perturbations to break the alignment between what a WAM imagines and what it executes. BadWAM characterizes this attack surface along two natural criteria: attack strength and stealthiness. When the adversary prioritizes disruption, BadWAM instantiates an **action-only adversarial attack**, which directly drives the model toward task-failing actions. When the adversary additionally prioritizes stealth, BadWAM instantiates an **imagination-preserving adversarial attack**, which seeks to induce harmful action shifts while keeping the model's predicted future close to its clean imagination. Together, these two attacks capture a spectrum of WAM-specific failures: from overt action hijacking to stealthier cases where the model appears to imagine a plausible future but executes a desynchronized action. We evaluate BadWAM across multiple WAM variants and benchmarks. Results show that our attacks substantially reduce task success rates under different execution setups. For example, our action-only attack reduces the model performance from 96.5% to 43.1% success on LIBERO. The results of our imagination-preserving attack further exposes a WAM-specific vulnerability: moderate future-preserving regularization can maintain strong attack performance while reducing future imagination drift.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.