- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper focuses on improving the inference latency of World Action Models (WAMs). Unlike previous works that focus on architectural changes, this paper proposes coordinating parallel execution with adaptive computation to achieve low-latency inference across diverse WAM architectures, resulting in a training-free framework called RealtimeWAM.| On an RTX 4090, RealtimeWAM leads to approximate speedups of 8.90× and 10.67× in measured mean inference latencies for FastWAM and OpenWAM. The average success rates are comparable to those of native inference, respectively, within 0.02 and 0.53 percentage points.
Critical strengths
-
This work proposes to improve the inference latency of WAMs at the execution level, which can speed up diverse WAMs without the need for of changing architectures. This infra level improvement can (nearly) universally benefit many existing and future WAM designs.
-
The significant speedups with negligible performance drops, evidenced by the experiments, are pronounced.
Critical weaknesses
- There are three major proposed techniques: 1). Dependency-aware parallel execution; 2). Hardware-aware token reuse; 3). Motion-adaptive refinement. Table 2 provides the only latency in a bottom-up manner from the native inference. The ablation studies provide both latency and success rate in a top-down manner from the full proposed method. A bottom-up ablation study providing both latency and success rate can largely strengthen the paper, which can provide a better understanding of the success-latency trade-off for each component. From my point of view, the first component seems lossless, while the other two rely on some hypotheses. The extra ablation can help me better understand the proposed techniques.
Questions for authors
-
Please read the weaknesses
-
I will also be interested in the success-latency performance under the distribution shift. For example, training on LIBERO and evaluating on LIBERO-Plus will be a good benchmark. Can you also add the extra evaluation on LIBERO-Plus with the existing checkpoints on LIBERO? (at least for RealtimeWAM)
We thank the reviewer for recognizing the value of execution-level acceleration. The requested bottom-up ablation helps clarify each component's contribution; we report it below together with the LIBERO-Plus protocol. Below, F denotes the number of full Transformer evaluations within the ten denoising steps.
W1. Bottom-up ablation with both success rate and latency.
Starting from native OpenWAM inference, we add compilation and parallel execution at a fixed 10F budget, and then add observation-token FFN reuse and adaptive computation separately and in combination:
The last three rows build on parallel execution. Latencies average the four evaluation settings; on RoboTwin alone, the last two rows take 72.10 / 63.76 ms (paper Table 3). Compilation and parallel execution keep the computation unchanged and match native outputs to numerical precision (paper Table 8), so they share the native success rate.
Parallel execution gives a 2.02× speedup. FFN reuse cuts first-forward latency from 37.12 to 31.62 ms at about a 0.4-point drop; adaptive computation lowers the average number of evaluations to about 2.8 with almost no change in success. For reference, fixed 2F and fixed 4F give 89.62% / 54.70 ms and 93.68% / 95.26 ms.
Q2. Evaluating the LIBERO checkpoints on LIBERO-Plus.
The paper's LIBERO-Plus results already follow this protocol: we directly evaluate the LIBERO-finetuned checkpoints with all weights frozen under distribution shift.
RealtimeWAM changes success by +1.40 and −0.90 points relative to native inference.