SparseWAM: Relevance-Guided Asymmetric Sparsification for World Action Models

World Action Models (WAMs) use world-model representations to condition action generation, but compressing these representations can compromise closedloop robustness. We study where sparsification should be applied, how task relevance should guide token selection, and how world-side compression interacts with action-side acceleration. Our analysis shows that sparsifying exported conditioning representations after dense world-model computation preserves task success substantially better than pruning visual tokens before representation formation, while aggressive diversity-based pruning becomes fragile under distribution shift. Motivated by these observations, we introduce Sparse- WAM, a training-free framework for relevance-guided asymmetric sparsification. On the world side, SparseWAM follows a relevance-before-redundancy principle: instruction-conditioned scores first restrict the candidate set, after which diversity-based selection enforces a fixed conditioning budget. On the action side, it adapts similarity-based layer skipping to selected Action-DiT denoising steps. At 50% conditioning retention with ActionSkip, SparseWAM achieves 95.00% on LIBERO and 50.66% on LIBERO-Plus with 369.20 ms inference latency, compared with 455.73 ms for the dense model and 378.40 ms for ActionSkip alone. Under a more aggressive 25% budget, semantic guidance improves LIBEROPlus success from 42.83% to 44.16% with world-side sparsification alone and from 42.50% to 44.22% when combined with ActionSkip, partially recovering the degradation caused by compression. Controlled real-robot shirt-folding trials further show that moderate sparsification retains more successful executions than aggressive joint compression. These results identify compression location and token-selection strategy as important factors in the efficiency–robustness trade-off of WAM inference.