acceptodds
Withdrawn from ICLR 2027

WAMachine: Training-Free Acceleration of World Action Models via Stateful Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

10 views

Abstract

World Action Models (WAMs) enable future-aware control by jointly modeling actions and environment dynamics. However, iterative diffusion or flow inference incurs substantial denoising latency. Prior inference state offers a natural opportunity for acceleration, yet changing planning contexts, observations, and intermediate representations can quickly render retained state stale. Preserving useful computation therefore requires adapting inference state rather than reusing it as-is. To this end, we present , a training-free framework that accelerates WAM inference by preserving and adapting inference state for efficient and accurate continuation as the control loop evolves. Across closed-loop replans, Trajectory Remapping remaps replan state from the preceding replan to initialize the next replan, reducing redundant trajectory generation. Across denoising steps, Observation Rebinding performs anticipatory inference during action execution and rebinds retained denoising state to the real observation for continuation when consistency checks pass, reducing latency exposed to the control loop. Across Transformer layers, Residual Rescaling selectively rescales retained layer state and refreshes it through full computation of the middle layers when probe checks fail, reducing repeated Transformer computation. Evaluations of three representative WAM architectures on LIBERO and RoboTwin 2.0 show that achieves 1.47–3.05 speedups in observation-to-action latency and 2.23–3.27 speedups in GPU inference time per replan, while preserving 96.69–99.54% of native WAM task success.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
world-action models, embodied AI, stateful inference, inference acceleration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ANDHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper targets inference latency in World Action Models (e.g., Cosmos Policy, Fast-WAM, Motus), where closed-loop control repeatedly recomputes closely related states across three nested loops: replanning, diffusion/flow denoising, and DiT layers. It proposes WAMachine, a training-free framework with one mechanism per loop: Trajectory Remapping warm-starts each replan from the previous solution; Observation Rebinding runs a few denoising steps during action execution using the model’s predicted future observation, then swaps in the real observation if a consistency check passes; and Residual Rescaling skips middle DiT layers by rescaling a cached residual, guided by a shallow probe. On LIBERO and RoboTwin 2.0, the authors report 2.23–3.27× lower GPU time per replan and 1.47–3.05× lower observation-to-action latency, while retaining 96.7–99.5% of native task success.

Critical strengths

The problem is practical, and the evaluation is careful about it. Reporting GPU time and O2A latency separately makes it clear how much of the gain comes from saved computation versus latency hidden by overlapping with execution. The timing definitions in Appendix A are precise.

The evaluation is broad and fairly honest: three WAMs with different architectures, large-sample success evaluation (6,000 / 2,000 episodes), and a separate measurement of the CUDA Graphs contribution (about 2.5× speedup remains without it).

Critical weaknesses

Limited novelty. Each component closely mirrors existing work: Trajectory Remapping resembles the warm starts in RTI-DP and STEP, Observation Rebinding resembles RTC/FutureRTC, and Residual Rescaling resembles DiCache, TeaCache, and C3ache. The related work section acknowledges this. The “state continuity” framing reads more like packaging than a new idea, and the paper does not make clear what it contributes beyond combining these techniques. DiCache and C3ache, the closest competitors, are also not included as baselines.

Simulation only. Observation Rebinding relies on pixel-level RMSE between predicted and real observations, which is likely to be much less reliable under real-world lighting, noise, and occlusion; even in simulation, the acceptance rate on Motus is only 54%. Since closed-loop latency is the main selling point, the lack of any real-robot experiment weakens the claims. End-to-end episode speedup is also modest (1.03–1.29×), and the Motus setup uses an extra rendering GPU.

Questions for authors

According to Table 5, the refresh path already uses fewer denoising steps than native inference (e.g., 3 vs. 5 for Cosmos Policy, 7 vs. 10 for Motus). How does native inference with the same reduced number of steps compare in speed and success rate? This baseline is needed to tell how much of the speedup comes from Trajectory Remapping itself rather than simply running fewer steps.

Many hyperparameters are set per model (layer boundaries, α range, cosine and fitting thresholds, OR thresholds, entry noise level) and were chosen through “diagnostic experiments.” Were these tuned on the evaluation tasks? How sensitive are speed and success rate to them, and what would it take to apply WAMachine to a new WAM?

2 comments

Thank you for the detailed feedback.

To W1: Novelty and missing baselines.

1. System design. Our contribution is a unified stateful inference system that abstracts replanning, denoising, and Transformer execution into three nested loops. It coordinates state retention, adaptation, and refresh across loops, rather than simply combining accelerators.

Sec. 3.4 explains: TR initializes OR, while RR checks layer reuse in anticipatory and real-conditioned inference. OR rejection preserves TR initialization; after OR acceptance, RR independently checks layer reuse.

2. Comparisons. We added DiCache/C³ache comparisons on fixed 200-episode subsets. Entries give GPU ms / O2A ms / SR (%).

ModelDiCacheC³acheWAMachine
Cosmos Policy167.32 / 528.22 / 95.0139.67 / 495.98 / 96.0120.29 / 434.64 / 96.0
Fast-WAM-IDM237.87 / 286.23 / 95.5233.21 / 281.07 / 96.5112.52 / 136.42 / 97.0
Motus1010.67 / 1898.56 / 82.0856.78 / 1754.34 / 82.0652.26 / 906.38 / 82.0

WAMachine achieves lower latency with equal or higher SR. Separately, Table 3 shows Fast-WAM-IDM O2A increasing from 144.75 ms to 177.20/165.62/205.45 ms without TR/OR/RR, respectively.

To W2: Observation robustness, simulation, and hardware.

  • Observation checks and execution timing. We apologize for the unclear description. Cosmos Policy and Motus check both images and robot state, with state carrying half the weight (Appendix B.3). Visual disturbances affect the image term, while continuation depends on the combined score. Motus accepts 54.34% of launched prefixes, covering 38.67% of noninitial replans. With κmax⁡=1\kappa_{\max}=1, an accepted prefix is followed by a scheduled refresh. A mismatch instead restarts from TR initialization under real observations. Motus achieves 2.23× GPU / 2.59× O2A speedups, including rejected work. Across 200 Cosmos Policy episodes, the one-step prefix of the 1+2 schedule has P95 latency of 79.0 ms with Graphs / 148.7 ms eager, fitting within the nominal 800 ms for 16 actions at 20 Hz. This supports computational feasibility during physical execution on this hardware.

  • E2E speedup. Appendix D.1/Table 8 report full-episode timing. O2A sums real-observation-to-CPU-action-readiness intervals; the rest includes setup, execution, simulation, rendering, and observation waits. Native/WAMachine O2A shares are 61.6/51.7%, 55.3/25.0%, 17.5/7.6% for Cosmos Policy/Fast-WAM-IDM/Motus. The 1.03–1.29× all-episode speedups reflect this non-O2A time and additional failures running to episode limits. For episodes where both methods succeed, Table 8 reports 1.22×/1.30×/1.12× full-episode speedups.

  • Rendering GPU. We sincerely apologize for the inaccurate description. All Motus methods use one policy GPU plus one rendering GPU. We will correct Section 4.1, Table 1, and Appendix B.4.

To Q1: Reduced-step Native versus TR.

We disabled OR/RR for this matched 200-episode comparison. Native uses 5 steps for Cosmos Policy / 10 for Motus; reduced-step Native and TR both use 3 / 7, respectively. Entries give O2A ms / SR (%).

ModelOriginal NativeReduced-step NativeTR
Cosmos Policy636.77 / 98.0534.98 / 98.0541.01 / 99.0
Motus2343.14 / 84.01930.28 / 82.01976.22 / 83.0

Reducing steps lowers latency, and TR improves observed SR by 1 percentage point at the same budget on both models. This supports retained-state initialization for shorter inference.

To Q2: Calibration, sensitivity, and transfer.

  • Selection. Parameters were selected using a separate diagnostic set of 100 samples, with random seeds disjoint from those used in the final evaluation. The selected parameters were then fixed across tasks for each model/profile. We will clarify this in Appendix B.4.

  • Sensitivity. We varied one setting at a time on fixed 200-episode subsets:

    ModelConfigurationO2A msSR (%)
    Cosmos PolicyDefault434.6496.0
    Cosmos Policyα: [1,1.25] → [1,1.15]439.8296.0
    Cosmos PolicyRR cosine: 0.984 → 0.990442.7696.0
    Cosmos PolicyRR fitting error: 0.176 → 0.12446.8996.5
    Cosmos PolicyTR steps: 3 → 4482.2397.0
    Cosmos PolicySkipped layers: [4,24) → [4,20)451.7696.5
    MotusDefault OR τ = 0.10906.3882.0
    MotusOR τ: 0.10 → 0.07985.3282.5
    MotusOR τ: 0.10 → 0.13875.0781.0

    The Cosmos Policy RR variants change O2A by 1.2–3.9%, with SR at 96.0–96.5%. TR budgets and OR thresholds further adjust the speed–quality trade-off.

  • New models. We connect solver state, trajectory alignment, and conditioning, then select layer ranges, entry schedules, and thresholds through diagnostics and closed-loop evaluation. The stateful design requires no weight retraining.

Dear Reviewer ANDH,

We would like to kindly follow up on our rebuttal and ask whether you have any remaining concerns. We would be happy to provide further clarification or discuss any unresolved issues.

Reviewer_gBZnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes WAMachine, a training-free framework that accelerates inference of World Action Models (WAMs) by reusing intermediate computation across the three nested loops of closed-loop inference: replanning, iterative denoising, and DiT layers. The method is evaluated in simulation on Cosmos Policy and Fast-WAM-IDM (LIBERO) and Motus (RoboTwin 2.0).

Critical strengths

Practical, well-motivated problem. Iterative denoising latency is a real bottleneck for deploying WAMs in closed-loop control.

Components compose with explicit fallbacks.

Large-sample success evaluation. Table 2 uses 6,000 LIBERO episodes per model (3 seeds) and 2,000 RoboTwin 2.0 episodes, with matched initial states across methods.

Critical weaknesses

The success drop is real and consistent, but is framed as "preserved." On the 200-episode subsets in Table 1, the failure count roughly doubles on Cosmos Policy.

All experiments are in simulation, which is especially limiting for a latency-hiding method. The original papers of the accelerated models include real-robot evaluations. A real-robot experiment, needed to support the control-frequency claims.

Figure 3 is very dense and hard to read at print size; too many arrows and objects, with no focus.

Questions for authors

How exactly is the action-execution window simulated? Does simulation time advance during inference?

2 comments

Thank you for the careful reading. Your comments help us clarify the performance trade-off, execution timing, and presentation.

To W1: Success-rate degradation.

As you point out, on the 200-episode Cosmos Policy subset, SR falls from 98.0% to 96.0%, doubling failures from 4 to 8. We apologize for understating this cost and will replace unqualified “preserved” claims with an explicit efficiency–performance trade-off.

Balanced and Quality use tighter RR fitting-error thresholds and larger denoising budgets to recover success while retaining acceleration. Model weights and OR thresholds remain fixed; TR entry levels follow the schedules. Settings below follow Default → Balanced → Quality:

  • Cosmos Policy: RR threshold 0.176 → 0.12 → 0.08; steps 1+2 → 1+3 → 1+4 (Native: 5).
  • Fast-WAM-IDM: RR threshold 0.60 → 0.40 → 0.25; video steps 2+4 → 2+5 → 2+6, action steps 5 → 6 → 7 (Native: 10 video / 10 action).
  • Motus: RR threshold 0.40 → 0.30 → 0.20; steps 3+4 → 3+5 → 3+6 (Native: 10).

a+b denotes anticipatory + real-conditioned steps on the accepted-prefix path. Entries are SR (%) / O2A speedup over Native.

ModelNative SRDefaultBalancedQuality
Cosmos Policy98.0597.02 / 1.47×97.73 / 1.33×97.98 / 1.20×
Fast-WAM-IDM98.6098.15 / 3.05×98.40 / 2.49×98.58 / 2.06×
Motus86.1083.25 / 2.59×84.50 / 2.14×85.55 / 1.82×

SR uses 6,000/6,000/2,000 episodes per profile, respectively; timing uses fixed 200-episode subsets. Quality reduces the SR gaps from 1.03/0.45/2.85 to 0.07/0.02/0.55 percentage points, retaining 1.20×/2.06×/1.82× O2A speedups.

To W2: Real-robot validation and control-frequency claims.

We apologize for not clearly distinguishing measured inference gains from their implications for physical control frequency.

  • Measured gains. GPU inference time counts all DiT work, including anticipatory computation, rejected prefixes, and refreshes. The 2.23–3.27× speedups therefore reflect reduced computation time; O2A additionally benefits from overlap. These results include CUDA Graphs. In the separate Fast-WAM-IDM ablation, WAMachine retains 2.53× GPU and 2.54× O2A speedups with Graphs disabled.

  • Execution feasibility. To assess whether anticipatory inference fits within physical action execution, we measured its latency on 200 Cosmos Policy episodes. The one-step prefix of the 1+2 schedule has P95 latency of 79.0 ms with Graphs / 148.7 ms in eager mode. Both fit well within the nominal 800 ms window for 16 actions at 20 Hz, supporting the computational feasibility of anticipatory inference during physical execution on the evaluated hardware. Figure 2(b)'s separate 2+3 diagnostic measures overlap with the actual simulation execution window.

To W3: Figure 3 readability.

We apologize for the difficulty in following Figure 3. It presents the three mechanisms and their interactions, accompanied by a detailed textual walkthrough in Secs. 3.1–3.4, which we hope clarifies the flow.

To Q1: How is the action-execution window simulated?

We execute each action chunk through environment steps while anticipatory inference runs concurrently. The overlap window is the chunk's measured wall-clock execution duration.

To Q2: Does simulation time advance during inference?

  • Simulation clock. Simulation advances through environment steps during action execution and pauses during foreground inference. After a new observation becomes available, the simulator waits for the next action chunk.

  • Timing scope. Accordingly, O2A runs from real-input availability before encoding to decoded-action readiness, including synchronization, rebinding, and prefix waits/restarts. Episode time additionally includes simulation, rendering, and execution.

Dear Reviewer gBZn,

We would like to kindly follow up on our rebuttal and ask whether you have any remaining concerns. We would be happy to provide further clarification or discuss any unresolved issues.

Reviewer_TcseOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This papers proposes a method to improve inference latency of WAM models. One of the issues of such models lies at inference speed, which significantly affects how it's applied for closed-loop policy execution. The proposed idea is to use caching mechanism to preserve useful computation. Different modules Observation Rebinding, Trajectory remapping, Residual Rescaling to reduce the discrepancy between previous cached plans and a new plan. Experiments are done on benchmarking simulation tasks LIBERO and RobotTwin 2.0 show this approach helps speed-up significantly in GPU inference time per replan, while preserving 96.69–99.54% of native WAM task success

Critical strengths
  • The authors propose three main mechanism to tackle the main technical problems of caching methods, which is sound and presented well. First where replanning through remapping between a previous plan and a new plan basically lies at using previously plan states as the new planning context, this would help denoising steps less explorative, and accelerate replanning. Secondly, in closed-loop settings, observation changes, therefore rebinding the observation by retaining previously denoised states can also improve computation. This is shown to easily pass consistency check. Third, the mechanism of DiT forward execution actually have smooth/continuous transition at the middle layers across adjacent denoising iterations, through which the authors propose using residual states for the middle layers to reduce DiT inference cost.

  • The proposed method is motivated well with the evidence and articulation of state continuity.

Critical weaknesses
  • Trajectory Remapping and Observation rebinding: The state continuity comes from various assumptions. The authors haven't made this point in a general way enough, i.e. assumptions when the proposed method/continuity breaks. For instance, high-level instruction might not change, however subtask/sub-motion during that main task changes, e.g. make a coffee is a main task, while pick a cup, then place the cup, press the button, they all might be annotated differently (very simple -> very expressive) this leads to different latent states of the (instruction, sub-instruction) observation, in particular at the switch of the sub-task. More generally, how this will also differ in the long-horizon tasks.

    • The experiments show the impact of the proposed method, in particular in inference time numbers, however the SR is still a bit lower than baselines, in particular those tasks are not known to be challenging. Therefore it's more helpful if the authors can further quantify this computation gain vs. any impact on policy performance.
Questions for authors

In overall, the paper addresses an important problem of WAM/VLA, where slow inference is a problem that limit closed-loop policy applications.

Q1: How the assumptions leading to the proposal, i.e. state continuity, be framed more formally to make WAMmachine well grounded on where it is applicable.

Q2: How the performance gap vs. baselines can be improved, i..e trade-off with the computation gain?

2 comments

Thank you for your questions about subtask transitions and the efficiency–performance trade-off.

To W1: Subtask transitions and long-horizon tasks.

As described in Sec. 1, state continuity means that inference states remain informative across replans, denoising steps, and layers while evolving with context. We evaluated Native, TR, and TR+OR on 10 LIBERO-Long tasks with Cosmos Policy using 50 matched initial conditions (150 rollouts). Boundaries were identified from simulator-defined goal events. At accelerated replans, an unexecuted Native solve under the same observation and instruction supplied the action-error reference.

  • Subtask boundaries. Near phase boundaries, action RMSE increased by 58% for TR and 51% for TR+OR relative to within-phase replans. Accepted-prefix coverage decreased from 75.6% to 60.8%, while observation-mismatch rejection increased from 4.9% to 21.5%. All three configurations achieved 50/50 success. This supports the effectiveness of OR's checks: more mismatched prefixes are rejected at subtask transitions.

  • Long-horizon tasks. The larger LIBERO-Long evaluation reports full-system WAMachine/Native SR of 95.47/97.53% for Cosmos Policy and 96.93/97.00% for Fast-WAM-IDM.

To W2: Computation gains versus policy performance.

We agree that latency gains should be presented alongside their effect on success. Default WAMachine's SR gaps are 1.03/0.45/2.85 percentage points for Cosmos Policy/Fast-WAM-IDM/Motus. We will revise “preserved performance” to state this trade-off explicitly. Q2 shows how increasing the computation budget narrows these gaps.

To Q1: Formal applicability conditions.

Fix the solver schedule, inference randomness, and task instruction gg. Let z=(x,h)z=(x,h) contain the trajectory latents xx at the entry noise level and the solver history hh needed for continuation. Let cc encode the real observation and robot state under the fixed instruction. Under cc, PcP_c advances the prefix and QcQ_c completes denoising and decoding; ac(z)=Qc(Pc(z))a_c(z)=Q_c(P_c(z)) is the resulting action chunk.

Assuming the denoiser, solver updates, and action decoder are continuous in their inputs, finite composition gives

\widetilde z\to z_{\mathrm{ref}}(c) &\implies a_c(\widetilde z)\to a_c(z_{\mathrm{ref}}(c)),\\\\ \widehat c\to c &\implies Q_c(P_{\widehat c}(\widetilde z))\to a_c(\widetilde z). \end{aligned}$$ - **TR:** $\widetilde z$ is the remapped initialization; $z_{\mathrm{ref}}(c)$ is the reference state at the same solver stage under the current condition. The premise is proximity after aligning the trajectory horizon and noise level, followed by refinement under the updated condition. - **OR:** $\widehat c$ is encoded from the predicted future, current robot state, and the same instruction $g$. The second relation fixes the TR initialization and changes only the prefix condition; continuation uses $c$. OR accepts prefixes when $S<\tau$ and $\kappa<\kappa_{\max}$; rejection restarts from TR initialization under the real observation. RR independently rechecks layer-state reuse after rebinding. In your subtask example, a transition can change state proximity or prediction accuracy despite unchanged instructions. TR provides refinement under the new condition, while OR's checks select continuation or refresh. The phase-boundary results in W1 examine this behavior. **To Q2: Reducing the performance gap.** Balanced and Quality use tighter RR fitting-error thresholds and larger denoising budgets to recover success while retaining acceleration. Model weights and OR thresholds remain fixed; TR entry levels follow the schedules. Settings below follow Default → Balanced → Quality: - **Cosmos Policy:** RR threshold 0.176 → 0.12 → 0.08; steps 1+2 → 1+3 → 1+4 (Native: 5). - **Fast-WAM-IDM:** RR threshold 0.60 → 0.40 → 0.25; video steps 2+4 → 2+5 → 2+6, action steps 5 → 6 → 7 (Native: 10 video / 10 action). - **Motus:** RR threshold 0.40 → 0.30 → 0.20; steps 3+4 → 3+5 → 3+6 (Native: 10). a+b denotes anticipatory + real-conditioned steps on the accepted-prefix path. Entries are SR (%) / O2A speedup over Native. | Model | Native SR | Default | Balanced | Quality | | --- | --- | --- | --- | --- | | Cosmos Policy | 98.05 | 97.02 / 1.47× | 97.73 / 1.33× | 97.98 / 1.20× | | Fast-WAM-IDM | 98.60 | 98.15 / 3.05× | 98.40 / 2.49× | 98.58 / 2.06× | | Motus | 86.10 | 83.25 / 2.59× | 84.50 / 2.14× | 85.55 / 1.82× | SR uses 6,000/6,000/2,000 episodes per profile, respectively; timing uses fixed 200-episode subsets. Quality reduces the gaps to 0.07/0.02/0.55 percentage points while retaining 1.20×/2.06×/1.82× O2A speedups, bringing success closer to Native.

Dear Reviewer Tcse,

We would like to kindly follow up on our rebuttal and ask whether you have any remaining concerns. We would be happy to provide further clarification or discuss any unresolved issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.