acceptodds
Withdrawn from ICLR 2027

Spec-WAM: Feedback Grounded Speculative Inference for World–Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

11 views

Abstract

World-action models (WAMs) leverage video-generation priors for robot action generation, but iterative denoising delays their response to new observations. Computing ahead during action execution can reduce this delay, yet predicted video and actions may become inconsistent with the observations that actually arrive. We introduce Spec-WAM, a training-free framework that tightly couples speculation across diffusion denoising and control loops. More specifically, during denoising sampling, extrapolation provides candidate action-denoising states for batched verification, amortizing model evaluation overhead. Within the control loop, the same pretrained WAM autoregressively prepares the next video-action trajectory while the robot executes its current action. After receiving the new observation, Spec-WAM checks whether the drafted video can be reused for action generation. For models that generate video before actions, it can reuse the accepted video together with its paired draft actions. For models that jointly denoise video and actions, a shared model query checks both streams, and action denoising continues from the last state that passes verification instead of restarting from noise. On RoboTwin 2.0 with unseen instructions, Spec-WAM matches the task success of OpenWAM's released inference on paired episodes (92.49% versus 92.65%) while significantly reducing mean feedback-to-action latency by 21.7%. On Next Forcing, it keeps task success and lowers this latency 34-fold, including the effect of fewer video denoising steps and of batched checking, which cuts sequential action-model calls from 51 to 4.02.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world-action models, speculative inference, robot learning, diffusion models, efficient inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_W7TrOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Spec-WAM, a training-free framework for reducing the feedback-to-action latency of world-action models. It speculates along two axes: •Within a decision, it extrapolates future action-denoising states and verifies them in one batched query. •Across decisions, a second GPU drafts the next video–action trajectory while the robot executes the current chunk. The draft is reused only if it passes a check under the real observation. The framework supports both video-then-action models (LingBot-VA, Next Forcing) and joint video–action models (OpenWAM-α). On RoboTwin 2.0, it substantially reduces mean latency (e.g., 5,010→147 ms on Next Forcing, 286→224 ms on OpenWAM-α), while keeping success within 1 pp of native inference.

Critical strengths

The theoretical treatment is rich and well developed: •The paper frames WAM inference along a denoising axis and an execution axis. •It gives a clean analysis of why extrapolated drafts can be accepted (Eq. 3). •It uses a defect-budget acceptance rule with an explicit bound on accumulated local error. •It grounds batched verification in a roofline cost model. The method is also training-free and applies to two different video–action coupling types. This makes it potentially useful across WAM architectures.

Critical weaknesses
  1. The experiments do not sufficiently support the main claim. •Most of the latency reduction comes from changes outside the speculative mechanism. On LingBot-VA and Next Forcing, reducing video denoising from 25 to 4 steps already brings latency from several seconds down to about 333 ms, with nearly unchanged success (Tables 4–5). Autoregressive drafting, the paper's main mechanism, adds only about 2× on top of this. The headline "34-fold" improvement is therefore measured against an untuned baseline. •The inference comparison is unfair. Spec-WAM uses two GPUs, whereas all baselines use one. The baselines could use the same extra compute, for example to run the acceleration methods discussed in the paper (parallel sampling, caching, or asynchronous execution). On OpenWAM-α, whose native program is already efficient, the gain with this extra compute is only 286→224 ms. •Key parameters lack justification and ablations. The paper gives no rationale or comparison experiments for τv = 0.2 (0.25 for LingBot-VA), σprobe = 0.95, σhold = 0.33, or τa = 0.1. It is unclear how these values were selected and how sensitive the results are to them. •There are no real-robot experiments. All results come from simulation, where the environment pauses during inference. Since the paper's contribution is latency reduction, its practical benefit should be demonstrated on real hardware.
  2. The presentation needs improvement. •The narrative logic and organization are unclear. The paper is dense with notation and cross-references. Several key implementation choices appear only in the appendix, such as returning drafted actions without checking them in the main configuration. •The figures appear hastily prepared. In several figures (e.g., Figures 1 and 2), text overlaps lines, arrows, or box borders, which hurts readability.
Questions for authors

1.What are the latency and success of native inference with 4 video steps and no speculation? This would isolate how much of the reduction comes from the speculative mechanism itself. 2.How were τv = 0.2 / 0.25, σprobe, σhold, and τa chosen? Please provide sensitivity or ablation results. 3.Under matched compute, how does Spec-WAM compare with baselines that use the second GPU for other acceleration methods? 4.Are there any real-robot results?

Reviewer_M9ihOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Spec-WAM is a training-free inference framework for pretrained world–action models. It speculates along two axes and commits a result only after checking it under the real observation:

Denoising axis: action-denoising states are drafted by extrapolating the clean estimate and checked in one batched query. Execution axis: while the robot executes the current chunk, a second GPU replica autoregressively drafts the next video and action trajectory. When feedback arrives, a video residual gate and an action-prefix check decide whether to reuse the draft or to fall back to recovery, which runs in parallel. On RoboTwin 2.0, success stays within 1 pp of native inference for LingBot-VA, Next Forcing and OpenWAM-α, while feedback-to-action latency drops (e.g., OpenWAM-α from 286 to 224 ms; Next Forcing from 5010 to 147 ms).

Critical strengths
  1. A clear systems formulation. Separating the denoising and execution axes, with a single residual covering both numerical draft error and conditioning change (Eq. 5), is clean. The adapter rules respect each model's video–action dependencies.

  2. Thorough evaluation. Three WAMs, 100 episodes per task on all 50 RoboTwin tasks, paired episodes, per-task tables, and an honest decomposition of where the latency comes from (Tab. 4/5).

  3. Useful component results. The batched action solver cuts sequential calls from 51 to 4.02 with success preserved (Tab. 6), and the verification ablation shows the gate matters (always-accept loses 2.3 pp on OpenWAM, Fig. 4).

Critical weaknesses

W1 [Experiments] Most of the speedup is not from speculation, and hardware is not matched. The "34×" mainly comes from cutting video steps from 25 to 4 plus the batched solver (5010 → 333 ms, Tab. 4/5), which needs no speculation. Speculation adds only ~2× more, and it uses a second GPU while all baselines use one. Compute-matched baselines and total compute/energy are missing.

W2 [Soundness] Latency is evaluated only in simulation. The benefit depends on drafts finishing within the execution window, yet drafts take up to 787 ms against a 640 ms action horizon (App. C). There is no real-robot or real-time evaluation for a latency-focused paper.

W3 [Novelty] The novelty is incremental. Speculative diffusion sampling (ParaDiGMS, De Bortoli et al.), speculative VLA inference with verification (FLASH, Spec-VLA), and pre-denoising during execution (DexWorldModel, ~50% latency reduction) already cover the main components. No comparison to DexWorldModel-style asynchronous speculation is given.

W4 Thresholds and shortcut. τ_v is tuned per model after inspecting archived residuals. On LingBot-VA and Next Forcing, drafted actions are returned without an action check (up to −1.83 pp). Tail latency (p95/p99) is not reported, although ~35% of decisions are rejected and take 261–320 ms.

Questions for authors
  1. What are the results against the 4-step native program with batched solver, and against a two-GPU native baseline, as the main reference? What are total GPU-seconds and energy per decision?

  2. How often are drafts not ready within a real execution window, and what latency and success result under real-time constraints or on a real robot?

  3. How does Spec-WAM compare to a DexWorldModel-style speculative asynchronous inference baseline?

Reviewer_FtDPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses the problem of robot response delay in WAMs, by developing a dual-axis drafter for future video and action chunk generation while executing the current actions. The observations after the execution are used to make a selection of the valid video and accept generated actions.

The speedup is mainly from the parallel drafting and action execution.

Critical strengths

The proposal of dual-axis drafting, and observation-based batch verification are valid and effective in speeding up the robot response.

Critical weaknesses
  • The sentences throughout the paper are hard to parse. Please consider using simple short sentences, not long clauses. For example, Lines 205-206 can be clearer if using multiple sentences.
  • The Caption for Figure 2 contain non-grammatical English
  • Figure 2 uses the same \textcircled{1} and \textcircled{2} twice to refer to different components in the diagram, which can cause confusion
  • Lines 054-081 contain detailed methodological information, which should not be in the introduction. It is too early for such details.
  • Lines 219-243, there are too many different variations of σ\sigma without definition, making the section difficult to verify for correctness.
Questions for authors

Explain the math notations and how it is related to the proposed framework.

How would adding one component at a time give a fair ablation of the component contribution? Is it possible to add each component independently?

What are the different ways of drafting? How would that affect the performance? Why dual drafting? Can you run ablation studies on single channel drafting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.