- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper aims to improve the inference latency of World Action Models (WAMs). It challenges whether action prediction requires full access to the world representation. Motivated by the case study performed by the authors, this paper proposes using a world-action interface to select and compress the video representations from the video backbone for the action decoder. With the world-action interface, the proposed RAPID-WAM can use a lightweight action module from HDP3, leading to an improvement in inference latency compared to Fast-WAM. The more complicated and heavier world-action interface will not slow down the inference speed, as all action denoising steps can share the same compressed condition, which is processed once from the world-action interface. The empirical results demonstrate that RAPID-WAM reaches improved performance with significantly reduced inference latency.
Critical strengths
-
This paper asks a critical question for WAMs: whether action prediction really requires access to the full world representation. This question and the paper's findings will benefit the community.
-
This paper proposes an interesting and effective solution: decoupling the world-action connection from action prediction, which are typically in the same action module in MoT-based WAMs.
-
The empirical performance is good, especially considering the improved inference latency.
Critical weaknesses
-
This paper verifies the proposed design on top of Fast-WAM (action-only, IDM, Joint), which should be viewed as the most important baseline. Fast-WAM is mainly evaluated on LIBERO and Robotwin in its original paper. However, this paper only shows the results on LIBERO, LIBERO-Plus, and RoboCasa. The omission of Robotwin raises some concerns about the empirical performance of RAPID-WAM.
-
The proposed technique seems sensitive to the choice of hyperparameters, such as the choice of readout layers (Table A6) and the number of queries (Table A7).
- For readout layers, it seems like the trends on LIBERO and LIBERO-Plus are different. And the manual selection of readout layers seems necessary, which slightly weakens the design of the proposed world-action interface.
- For query count, a similar observation.
- An in-depth discussion of the ablation study will significantly strengthen the paper.
-
The statements on the action parameters and FLOPs are not accurate (Sec. 5.3). In fact, RAPID-WAM assigns two functions of the original action expert in MoT-based WAMs, world-action connection and action generation, to the world-action interface and MLP-based action decoder. Therefore, the comparison of parameters and FLOPs between the action expert in FastWAM and the MLP-based action decoder in RAPID-WAM is a bit misleading. The counts of the world-action interface should be included. This discussion will also strengthen the paper:
- RAPID-WAM actually shifts some capacity and compute from the action expert/decoder to the world-action interface. However, the action expert/decoder needs repeated denoising steps, while the condition from the world-action interface only needs to be computed once. Such decoupling and shifting is critical to the inference speed.
(Very) Minor weaknesses
- The main paper can be more self-contained. For example,
- without searching in the appendix, I cannot know which dataset the efficiency evaluation is on.
- without searching in the appendix, I do not know what data the diagnosis of world-feature access was conducted on.
They can be made clear by adding a one-sentence description or a reference to the appendix.
- This paper holds a very similar high-level idea to the following paper:
- Faster WAM https://arxiv.org/abs/2608.02365v1
- Light WAM https://arxiv.org/abs/2606.08242 The authors are encouraged to include them in the related work discussion.
Note that I do not view this as a negative point in my assessment, as: 1). this paper proposes a different solution from them; 2). these papers are contemporaneous work and/or unpublished arXiv papers.
Questions for authors
-
Can you provide the experimental results on Robotwin under the same settings as in FastWAM? (This is the most critical)
-
Can you provide further discussion on the choice of readout layers and query counts?
-
Can the world-action interface benefit the more widely used DiT-based Action expert in terms of prediction performance and inference speed? The actionDiM based on HDP3 looks elegant but is less widely used at this time. I want to understand whether the gains in prediction performance and inference speed are mainly from HDP3 or the world-action interface design.
4 comments
Q3. Can the world–action interface improve the performance and inference speed of a DiT action expert?
Yes. Our additional experiments show that RAPID's world–action interface also substantially improves control performance and inference efficiency with ActionDiT. We have combined RAPID-WAM's world–action interface with ActionDiT to construct RAPID-WAM-DiT and evaluated its control performance and inference efficiency.
| Model variant | World–action interface | Action decoder | LIBERO | LIBERO-Plus | Total policy latency (speedup) |
|---|---|---|---|---|---|
| Fast-WAM | Native dense MoT | ActionDiT | 97.6% | 49.7% | 384.6 ms (1.00×) |
| RAPID-WAM-DiT | RAPID learned readout | ActionDiT | 98.3% | 78.0% | 154.0 ms (2.50×) |
| RAPID-WAM | RAPID learned readout | ActionDiM | 98.8% | 70.7% | 84.8 ms (4.53×) |
Compared with Fast-WAM, RAPID-WAM-DiT improves LIBERO-Plus success by 28.3 percentage points, while achieving a 2.5× end-to-end inference speedup. Notably, its success rate even exceeds that of the default RAPID-WAM (78.0% vs. 70.7%), indicating that the benefits of the RAPID interface do not depend on the lightweight ActionDiM [6] decoder.
We have added the RAPID-WAM-DiT results to Sec. 5.4 and provided the corresponding implementation details in the appendices.
Minor W1. The main text should specify the datasets used for efficiency evaluation and world-feature access diagnosis.
We thank the reviewer for pointing this out. Appendices A and E of the original manuscript already specify that both experiments are conducted on LIBERO. We have now also made this explicit in Sec. 3 / Fig. 2 and Sec. 5.3 / Table 6.
Minor W2. Please include Faster-WAM and Light-WAM in the related-work discussion.
We thank the reviewer for recommending these two valuable concurrent works. Faster-WAM [7] improves inference efficiency through cross-layer K/V fusion, while Light-WAM [8] combines cross-layer query pooling with lightweight modeling to reduce computational cost. Whereas these methods primarily focus on computational structure, RAPID-WAM further examines the more fundamental question of how much information action prediction needs to read from a world model. Through feature interventions, we find that a small subset of key world features is sufficient for effective control, motivating our compact, reusable world–action interface. This design has been validated across multiple WAM formulations and action decoders, achieving 98.8% success on LIBERO (98.5% for Faster-WAM and 97.2% for Light-WAM) and a 4.53× policy speedup (3.2× reported by Faster-WAM), while maintaining control performance on long-horizon and complex tasks. We have added the relevant discussion to Sec. 2.
References
[1] Tianxing Chen et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation arXiv preprint arXiv:2506.18088, 2025.
[2] Tianyuan Yuan et al. Fast-WAM: Do World Action Models Need Test-time Future Imagination? arXiv preprint arXiv:2603.16666v2, 2026.
[3] Mateo Clémente et al. La-JEPA: Compact Robot Policies with Latent World Modeling arXiv preprint arXiv:2611.08156, 2026.
[4] Teli Ma et al. DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control arXiv preprint arXiv:2603.10448, 2026.
[5] Mengqi Zhang et al. DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning arXiv preprint arXiv:2607.24159, 2026.
[6] Jinhao Zhang et al. Hyper-DP3: Frequency-Aware Right-Sizing of 3D Diffusion Policies for Visuomotor Control arXiv preprint arXiv:2605.01581v4, 2026.
[7] Liheng Ma et al. Faster-WAM: Do World Action Models Need Deep Action Modules? arXiv preprint arXiv:2608.02365v1, 2026.
[8] Ziang Li et al. Light-WAM: Efficient World Action Models with State-Fusion Action Decoding arXiv preprint arXiv:2606.08242, 2026.
Thanks for the rebuttal.
I believe the authors have addressed most of my concerns.
One remaining point is that FastWAM reports the results on full Robotwin training (Clean+Random training --> Clean+Random eval) . I believe the reported results aligning with the most important baseline in the paper will greatly strengthen the paper.
However, regarding the current rebuttal from the authors, I will raise my score accordingly.
We sincerely thank the reviewer for carefully considering our rebuttal and for the positive reassessment. We are glad that our additional experiments and clarifications have addressed most of your concerns. We greatly appreciate your constructive feedback, which has helped us further strengthen the paper.
We thank the reviewer for recognizing our research question, the decoupling of world–action interaction from action prediction, and RAPID-WAM's control performance and inference efficiency. We address the comments below.
On RoboTwin 2.0 [1], RAPID-WAM improves bimanual manipulation performance and zero-shot OOD generalization over Fast-WAM [2]. The original Fast-WAM paper trains on 27,500 demonstrations (2,500 clean + 25,000 randomized) with 64 GPUs. Given our limited computational resources, we made full use of the eight GPUs available in our lab during the rebuttal period (using four GPUs per training run) and conducted the controlled comparison using only 2,500 clean demonstrations under the clean-only setting of RoboTwin 2.0. Both models use identical training and evaluation settings:
† Faster-WAM results are from La-JEPA [3], Table 2.
On Clean→Clean, RAPID-WAM achieves 75.58%, 8.64 percentage points above Fast-WAM, also exceeding Faster-WAM's reported clean-only result (68.60%). On Clean→Random, zero-shot OOD success rises from 1.51% to 2.52%, a 66.9% relative improvement, also surpassing Faster-WAM's 2.20%. These results demonstrate improved bimanual control and generalization under a limited training budget. We have added the results and protocol details to Sec. 5.2 and Appendix C.
We thank the reviewer for this suggestion. Our readout layer positions are not manually selected for individual tasks; instead, we use a common uniform sampling strategy. DiT4DiT [4] finds that video features at different depths play different roles in control, while DeVA [5] also uses uniform sampling to access features from multiple layers. Inspired by these works, we select five evenly spaced layers (6, 12, 18, 24, and 30) from the 30-layer backbone, without selecting layer positions separately for different datasets. Tables A5 and A7 show that all configurations perform similarly on near-saturated LIBERO (97.6%–98.8%), while LIBERO-Plus is more sensitive to the layer count: success improves from 67.7% (K=1) to 70.0% (K=3) and 70.7% (K=5), but five and 30 layers perform nearly identically (70.7% vs. 70.8%). Thus, a few uniformly sampled depths suffice without elaborate layer selection. Query count trades performance against overhead: increasing Q from 2 to 8 improves LIBERO-Plus from 68.0% to 70.7%. Q=16 further raises it to 72.7%, but LIBERO falls from 98.8% to 97.7%, and readout computation increases. We therefore use K=5 and Q=8 by default to balance benchmark performance and interface cost. We have clarified this choice in Sec. 5.4.
We thank the reviewer for this suggestion. The original manuscript already reports the parameter counts of the decoder and readout interface separately in Appendix E, and the Action GFLOPs in the original Table 5 (Table 6 in the revision) already include interface computation. We have now further summarized the complete action-side costs in the main text and clarified how computation is reused (default action-only setting, N=10):
Although Fast-WAM caches world features, its action expert repeats dense world–action interactions at each denoising step, costing 257.26 GFLOPs across ten steps. RAPID-WAM instead reads and compresses world features once per observation (0.96 GFLOPs), then reuses the compact conditions across ten steps (about 0.011 GFLOPs per step for condition injection), for a total interface cost of 1.07 GFLOPs.
This read-once, reuse-across-steps design eliminates repeated dense interactions and enables lightweight decoding. Including the 6.01M readout parameters, RAPID-WAM reduces total action-side parameters by 65.1× and full action-stage FLOPs by 268.6×, with a 4.53× end-to-end inference speedup. We have clarified reuse in Sec. 5.3, added interface parameters to Table 6 and totals for all three formulations to its caption, and added the default model's total parameter count to Appendix E.