- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper speeds up inference for world action models, which jointly generate future video and robot actions, while keeping explicit video prediction at test time. F2 Distill compresses a full-budget LingBot-VA teacher into two few-step students with different step budgets, using consistency distillation plus a distribution-matching loss on video. At inference, R2 Spec uses them as Draft and Target: the Target verifies the Draft's action proposals, locally repairs some rejections, and otherwise falls back to full Target generation. On RoboTwin 2.0 the system roughly matches the teacher and is about 17 times faster per call, although the paper notes that this ratio is not normalized for hardware. LIBERO and four real-robot tasks are also evaluated.
Critical strengths
-
F2 Distill mostly builds on Flash-WAM's consistency setup, adding an adapted DMD loss on video, but it works well. At the same step budgets, both students beat Flash-WAM on RoboTwin 2.0 Clean success (91.5 vs. 88.4 and 92.5 vs. 90.2) and on FVD and LPIPS (Table 2), and the full system ends up on par with the teacher. The paper does not say whether the Flash-WAM success rates in Tables 1 and 2 were obtained with the authors' own RoboTwin protocol; Sec. 4.1 only says that "Baseline rows may use other denominators".
-
The real-robot study is an appropriate test for a method that cuts latency. The four AgileX Piper tasks are dynamic or time-critical, such as catching a block sliding off a box or grasping a rolling ball, and each method gets 100 trials per task. DiSpecWAM averages about 80% against 72.5% for Fast-WAM, the strongest baseline. The paper gives few details on how these runs were set up, for example the training data, the baselines' step budgets or the latency on the robot.
Critical weaknesses
-
My main concern, and the main reason for my score, is the motivation. The paper claims that generating future video at inference helps control (p. 1, Fig. 1b–c), but never tests this. It even says on p. 2 that "omitting generated future frames does not imply losing learned dynamics or closed-loop control." In DiSpecWAM the video only conditions the actions. Draft video is discarded and the verifier compares actions only, so I consider the benefit of faster video generation unclear. Beating Fast-WAM, a different model that skips test-time video generation, does not resolve this question. The success drop without the FMDD video loss is equally consistent with video prediction acting only as a training-time regularizer. Without the video argument, the difference from Realtime-VLA FLASH stated in Sec. 2 reduces to local repair with a recheck, which adds 0.70 points and saves 51 ms per call (Table 4).
-
I am also not convinced by the gains credited to R2 Spec. The 205 ms is per inference call, "not compute time per executed action" (Sec. 4.2), since R2 Spec may execute only 8 of 32 actions before replanning. Target-only takes 376 ms per 32-action chunk. If it executes the whole chunk, DiSpecWAM must average more than about 17 executed actions per call to be cheaper per action. The paper does not report this number. Nor can I reconcile the timing. One extra action denoising step costs 27 ms (Table 3), yet the full system is only 22 ms slower than Draft-only. Every call still runs a Target verification. The 1-point gain over Target-only is small and, as the authors acknowledge in Sec. 4.5, confounded with replanning frequency.
Questions for authors
Q1. Beyond conditioning the actions, what is the generated future video used for at inference, and what evidence shows that generating it at test time improves control?
Q2. What do Draft-only and Target-only achieve if they also replan after the first 8 actions, as DiSpecWAM does whenever it accepts only a prefix or a repair?
3 comments
W2 & Q2: Matched 8-Step Replanning, Per-Action Cost Break-Even, and Timing Reconciliation
Thank you for this precise quantitative audit. We address matched 8-step replanning, the 17.4-action break-even vs. Target-only, and why total latency is 205 ms (+22 ms over Draft-only vs. +27 ms in Table 3) below:
(1) Draft-only and Target-only with 8-action replanning (). We evaluated Draft-only and Target-only when generating and executing 8 actions per call before re-observing and replanning:
| Method / Variant | Steps (v/a) | Exec. Prefix | Clean SR (%) | ms / call | ms / action |
|---|---|---|---|---|---|
| Full LingBot-VA (Teacher) | 25 / 50 | 32 | 92.90 | 3404.85 | 106.40 |
| Draft-only | 1 / 2 | 32 | 91.48 | 183.00 | 5.72 |
| Draft-only (8-step replan) | 1 / 2 | 8 | 91.60 | 176.00 | 22.00 |
| Target-only | 2 / 4 | 32 | 92.50 | 376.00 | 11.75 |
| Target-only (8-step replan) | 2 / 4 | 8 | 93.12 | 354.00 | 44.25 |
| DiSpecWAM (with Repair) | 1/2 & 2/4 | Adaptive (8 / 32) | 93.50 | 205.00 | 7.67 |
RoboTwin 2.0 Clean (50 tasks). The 8-step baselines generate and execute 8 actions per call before re-observing and replanning. Both generation length and replanning frequency change relative to the 32-step baselines.
Generating and executing 8 actions per call yields points for Draft-only (91.60%) and points for Target-only (93.12%, surpassing 32-step open-loop Full's 92.90%), but raises Target-only's cost to 44.25 ms/action. DiSpecWAM reaches 93.50% ( points above 8-step Target-only) at 7.67 ms/action ( faster), as -bounded repair preserves sharp contact commitment while triggering 8-step replanning selectively upon cross-budget disagreement.
(2) Executed actions per call vs. the 17.4-action break-even point. Your break-even math is exact: since 32-step Target-only costs ms/action, DiSpecWAM at 205 ms/call must average over executed actions/call. From Table 5 (), final return routes are 74% Draft (32 actions), 22% Repair (8 actions), and 4% Fallback (32 actions), averaging executed actions/call---well above 17.45, giving 7.67 ms/executed action ( faster than 32-step Target-only and faster than Full LingBot-VA):
| Final Return Route () | Call Share | Executed Actions |
|---|---|---|
| Accepted Draft | 74% | 32 |
| Accepted Local Repair | 22% | 8 |
| Full Target Fallback | 4% | 32 |
| Weighted Mean per Call | 100% | 26.72 |
| Method | ms / call | Actions / call | ms / action | Speedup |
|---|---|---|---|---|
| Full LingBot-VA (25/50, 32-step) | 3404.85 | 32.00 | 106.40 | 1.00$\times$ |
| Target-only (2/4, 8-step replan) | 354.00 | 8.00 | 44.25 | 2.40$\times$ |
| Target-only (2/4, 32-step) | 376.00 | 32.00 | 11.75 | 9.06$\times$ |
| Draft-only (1/2, 32-step) | 183.00 | 32.00 | 5.72 | 18.60$\times$ |
| DiSpecWAM (Adaptive 8 / 32) | 205.00 | 26.72 | 7.67 | 13.87$\times$ |
(3) Timing reconciliation: Why +1 step in Table 3 costs 27 ms, whereas verification + recovery adds only 22 ms (183 205 ms).
-
Why Table 3's extra step costs 27 ms: Moving from
(1, 1)(156 ms) to(1, 2)(183 ms) adds a sequential cross-modal Transformer step, whereas most of the 156 ms base latency comes from multi-view RGB encoding and full-sequence video denoising. -
Why
Verifycosts far less than 27 ms: Initial verification is not a sequential cross-modal step. On every call, it reuses already-encoded current observations and cached Target video features (skipping Target video generation on all 74% accepted calls) and evaluates both noise levels in one GPU-batched action-branch forward pass, costing only a few milliseconds. -
Why amortized recovery adds only $\sim$16 ms (saving 51 ms over
w/o Repair): Fresh Target context runs on only the 26% rejected calls. Without repair (w/o Repair, Table 4), all 26% rejections run full 4-step Target generation, raising mean latency to 256 ms ( ms over Draft-only). Replacing 22% of those full rollouts with 1-step Euler repair saves 51 ms on average ( ms), reconciling Tables 3--5.
Protocol details. Section 4.1/4.3 also clarifies that Table 2 evaluates Flash-WAM on the identical LingBot-VA backbone, 25k RoboTwin dataset, and protocol, and reports real-robot fine-tuning details alongside the 400-trial PFD comparison (79.75% vs. 74.00%).
Thank you for the rebuttal and the new per-action and 8-step results. The per-action accounting resolves my question on cost: at 26.72 executed actions per call, DiSpecWAM is at 7.67 ms per action against 11.75 ms for Target-only. On success, however, Target-only with 8-step replanning already reaches 93.12%, so only 0.38 of the 1.0-point gain over Target-only remains.
My main concern about the motivation is not resolved. The same-checkpoint toggle is run on HumanWAM, a model that is not in the paper and not described in the response, rather than on DiSpecWAM. Switching off an input the model was trained to use is also expected to hurt, so this test does not separate the test-time role of the video from its role in training. The Fast-WAM and PFD comparisons still involve different models, and the FMDD ablation changes training, not inference.
I appreciate the additional experiments, but I still consider the benefit of generating video at test time not shown for DiSpecWAM, so I am keeping my score.
We sincerely thank the reviewer for the sharp, focused evaluation of our test-time video motivation, timing accounting, and controlled comparisons. We address each concern from both architectural principles and controlled empirical evidence below.
W1 & Q1: Role of Test-Time Future Video in Control and Verification
Thank you for raising this foundational question. We clarify both (1) what generated future video actually does inside DiSpecWAM at inference (beyond conditioning Draft actions) and (2) controlled evidence isolating test-time video generation from training-time regularization:
(1) Architectural role: Future video anchors Draft proposals, Target verification, and local repair. First, we apologize that our phrasing on page 2 (“omitting generated future frames does not imply losing learned dynamics…”) read as undercutting our motivation; it was intended only to note that video-skipping policies remain reactive, and we revised Section 1 accordingly. Second, in DiSpecWAM, generated future video is not discarded without affecting verification:
Action proposal conditioning: Inside the 30-layer causal Transformer, Draft action queries attend directly to generated future-video latents V^td to anticipate multi-view contact geometry before committing actions (discarding V^td after A^td is produced only means predicted frames are not committed into observation history).
Target verification and repair conditioning (Eqs. 9, 12--13): Rather than comparing actions in isolation, the Target verifier's velocity field vTa(A~t,σk,σk,ct−) (Eq. 9), scene-shift detector efresh=Distcont(A^T,t,1:Hr−,A^T,t,1:Hrf) (Eq. 12), and one-step Euler repair (Eq. 13) are all conditioned on ct− and ctf---which are precisely the future-video representations generated at test time by the Target video branch. Degrading few-step video directly harms control: removing video FMDD (
w/o Distribution Alignment, Table 4) drops Clean SR by 9.33 points (93.50% → 84.17%), and removing video consistency while keeping action consistency drops LIBERO SR by 8.20 points (98.45% → 90.25%).(2) Controlled evidence that test-time video generation improves control (ruling out training-only regularization).
Distinction from Realtime-VLA FLASH. Beyond retaining calibrated test-time future video via F$^2$Distill, R$^2$Spec decouples expensive multi-view Target video generation from verification via cached ct− and efresh re-anchoring, and replaces binary fallback with κ-bounded Euler repair (+0.70 SR and −51 ms over FLASH-style
w/o Repair).