- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper targets inference latency in World Action Models (e.g., Cosmos Policy, Fast-WAM, Motus), where closed-loop control repeatedly recomputes closely related states across three nested loops: replanning, diffusion/flow denoising, and DiT layers. It proposes WAMachine, a training-free framework with one mechanism per loop: Trajectory Remapping warm-starts each replan from the previous solution; Observation Rebinding runs a few denoising steps during action execution using the model’s predicted future observation, then swaps in the real observation if a consistency check passes; and Residual Rescaling skips middle DiT layers by rescaling a cached residual, guided by a shallow probe. On LIBERO and RoboTwin 2.0, the authors report 2.23–3.27× lower GPU time per replan and 1.47–3.05× lower observation-to-action latency, while retaining 96.7–99.5% of native task success.
Critical strengths
The problem is practical, and the evaluation is careful about it. Reporting GPU time and O2A latency separately makes it clear how much of the gain comes from saved computation versus latency hidden by overlapping with execution. The timing definitions in Appendix A are precise.
The evaluation is broad and fairly honest: three WAMs with different architectures, large-sample success evaluation (6,000 / 2,000 episodes), and a separate measurement of the CUDA Graphs contribution (about 2.5× speedup remains without it).
Critical weaknesses
Limited novelty. Each component closely mirrors existing work: Trajectory Remapping resembles the warm starts in RTI-DP and STEP, Observation Rebinding resembles RTC/FutureRTC, and Residual Rescaling resembles DiCache, TeaCache, and C3ache. The related work section acknowledges this. The “state continuity” framing reads more like packaging than a new idea, and the paper does not make clear what it contributes beyond combining these techniques. DiCache and C3ache, the closest competitors, are also not included as baselines.
Simulation only. Observation Rebinding relies on pixel-level RMSE between predicted and real observations, which is likely to be much less reliable under real-world lighting, noise, and occlusion; even in simulation, the acceptance rate on Motus is only 54%. Since closed-loop latency is the main selling point, the lack of any real-robot experiment weakens the claims. End-to-end episode speedup is also modest (1.03–1.29×), and the Motus setup uses an extra rendering GPU.
Questions for authors
According to Table 5, the refresh path already uses fewer denoising steps than native inference (e.g., 3 vs. 5 for Cosmos Policy, 7 vs. 10 for Motus). How does native inference with the same reduced number of steps compare in speed and success rate? This baseline is needed to tell how much of the speedup comes from Trajectory Remapping itself rather than simply running fewer steps.
Many hyperparameters are set per model (layer boundaries, α range, cosine and fitting thresholds, OR thresholds, entry noise level) and were chosen through “diagnostic experiments.” Were these tuned on the evaluation tasks? How sensitive are speed and success rate to them, and what would it take to apply WAMachine to a new WAM?
2 comments
Dear Reviewer ANDH,
We would like to kindly follow up on our rebuttal and ask whether you have any remaining concerns. We would be happy to provide further clarification or discuss any unresolved issues.
Thank you for the detailed feedback.
To W1: Novelty and missing baselines.
1. System design. Our contribution is a unified stateful inference system that abstracts replanning, denoising, and Transformer execution into three nested loops. It coordinates state retention, adaptation, and refresh across loops, rather than simply combining accelerators.
Sec. 3.4 explains: TR initializes OR, while RR checks layer reuse in anticipatory and real-conditioned inference. OR rejection preserves TR initialization; after OR acceptance, RR independently checks layer reuse.
2. Comparisons. We added DiCache/C³ache comparisons on fixed 200-episode subsets. Entries give GPU ms / O2A ms / SR (%).
WAMachine achieves lower latency with equal or higher SR. Separately, Table 3 shows Fast-WAM-IDM O2A increasing from 144.75 ms to 177.20/165.62/205.45 ms without TR/OR/RR, respectively.
To W2: Observation robustness, simulation, and hardware.
Observation checks and execution timing. We apologize for the unclear description. Cosmos Policy and Motus check both images and robot state, with state carrying half the weight (Appendix B.3). Visual disturbances affect the image term, while continuation depends on the combined score. Motus accepts 54.34% of launched prefixes, covering 38.67% of noninitial replans. With κmax=1, an accepted prefix is followed by a scheduled refresh. A mismatch instead restarts from TR initialization under real observations. Motus achieves 2.23× GPU / 2.59× O2A speedups, including rejected work. Across 200 Cosmos Policy episodes, the one-step prefix of the 1+2 schedule has P95 latency of 79.0 ms with Graphs / 148.7 ms eager, fitting within the nominal 800 ms for 16 actions at 20 Hz. This supports computational feasibility during physical execution on this hardware.
E2E speedup. Appendix D.1/Table 8 report full-episode timing. O2A sums real-observation-to-CPU-action-readiness intervals; the rest includes setup, execution, simulation, rendering, and observation waits. Native/WAMachine O2A shares are 61.6/51.7%, 55.3/25.0%, 17.5/7.6% for Cosmos Policy/Fast-WAM-IDM/Motus. The 1.03–1.29× all-episode speedups reflect this non-O2A time and additional failures running to episode limits. For episodes where both methods succeed, Table 8 reports 1.22×/1.30×/1.12× full-episode speedups.
Rendering GPU. We sincerely apologize for the inaccurate description. All Motus methods use one policy GPU plus one rendering GPU. We will correct Section 4.1, Table 1, and Appendix B.4.
To Q1: Reduced-step Native versus TR.
We disabled OR/RR for this matched 200-episode comparison. Native uses 5 steps for Cosmos Policy / 10 for Motus; reduced-step Native and TR both use 3 / 7, respectively. Entries give O2A ms / SR (%).
Reducing steps lowers latency, and TR improves observed SR by 1 percentage point at the same budget on both models. This supports retained-state initialization for shorter inference.
To Q2: Calibration, sensitivity, and transfer.
Selection. Parameters were selected using a separate diagnostic set of 100 samples, with random seeds disjoint from those used in the final evaluation. The selected parameters were then fixed across tasks for each model/profile. We will clarify this in Appendix B.4.
Sensitivity. We varied one setting at a time on fixed 200-episode subsets:
The Cosmos Policy RR variants change O2A by 1.2–3.9%, with SR at 96.0–96.5%. TR budgets and OR thresholds further adjust the speed–quality trade-off.
New models. We connect solver state, trajectory alignment, and conditioning, then select layer ranges, entry schedules, and thresholds through diagnostics and closed-loop evaluation. The stateful design requires no weight retraining.