AWARE: Action Weak-spot Assessment before Robotic-world-model Execution
Abstract
Action-conditioned world models serve as environment surrogates for robotic planning, policy evaluation, and control, yet visually plausible predictions do not guarantee accurate modeling of action effects. Small action perturbations can induce response errors that propagate across successive prediction windows and ultimately produce substantial prediction drift. Existing visual and textual attacks conflate perceptual, semantic, and dynamics-prediction errors, making action-response errors difficult to isolate. We study the action-conditioning input as a separate attack surface and formalize pre-deployment action weak-spot discovery: identifying directions likely to induce large action-response mismatch between a world model and its environment without access to future environment responses. We propose AWARE (Action Weak-spot Assessment before Robotic-world-model Execution). Holding observation history and task semantics fixed, AWARE constructs a candidate pool under a shared budget in normalized action space and selects the candidate with the largest frozen-model response energy relative to the nominal prediction. Local response-geometry analysis characterizes the directional-alignment conditions and information limits under over-response and under-response. Compared with same-pool random selection, AWARE achieves higher mean true action-response mismatch at every tested budget: relative increases of – on CALVIN/Ctrl-World, – on RoboCasa/Ctrl-World, and – on CALVIN/IRA-SIM. AWARE provides a diagnostic of action–outcome consistency before deployment. Exploratory repair improves selected response measures but does not consistently reduce mismatch on a fixed episode-disjoint development set.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.