RAPID-WAM: Rethinking the World–Action Interface for Compact Action Decoding

World action models (WAMs) use the rich representations of video-pretrained world models for robot control, but existing methods typically rely on large action experts and dense world–action interactions. Through feature interventions across three representative WAM formulations, we find that selectively retaining approximately one quarter of the most action-sensitive world features preserves closed-loop performance close to full access. Based on this observation, we propose RAPID-WAM with a lightweight world–action interface that uses task-conditioned queries to read and compress representations from multiple world-model depths into compact control conditions before action decoding. RAPID-WAM outperforms the corresponding Fast-WAM baselines across action-only, inverse-dynamics, and joint world–action formulations and achieves state-of-the-art performance on multiple robot manipulation benchmarks. In the default action-only setting, RAPID-WAM reduces the world-condition representation delivered to action decoding by 11,760×, improves LIBERO-Plus success from 49.7% to 70.7%, and achieves a 4.53× speedup in end-to-end policy inference.