- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes STAIRCASE POLICY, a streaming inference framework for World-Action Models that aims to improve the efficiency of long action-chunk execution. It divides each chunk into sub-chunks that are progressively denoised and executed. During execution, new observations are used to refresh future latent predictions and refine the remaining actions, without repeatedly invoking the expensive VLA backbone. The authors also introduce a training procedure aligned with this inference schedule and explore prediction error for adaptive replanning. Experiments on LIBERO, LIBERO-Plus, DOMINO, and nine real-robot tasks show improved success rates and inference efficiency. The method is also evaluated across three VLA backbones.
Critical strengths
-
The paper addresses a practical issue in VLA deployment. Longer action chunks reduce inference overhead but suffer from stale observations. The proposed streaming schedule provides a reasonable way to incorporate new observations without repeatedly running the full backbone, offering a useful trade-off between efficiency and closed-loop responsiveness.
-
The experiments are fairly extensive, covering multiple benchmarks, three backbones, and real-robot evaluations. The improvements on dynamic real-robot tasks are particularly encouraging, with success increasing from 25.8% to 44.2% over the long-chunk baseline. The reported latency and training-cost analyses are also useful.
Critical weaknesses
-
The ablations do not sufficiently separate the contributions of staircase denoising, observation refresh, and future latent prediction. Since the action expert already receives updated visual features, it is unclear how much additional benefit comes from predicting future latents. The streaming-aligned training procedure also introduces several changes simultaneously. Without isolating these components, the necessity of the proposed world-model mechanism remains unclear.
-
The novelty over existing streaming and reactive VLA methods is not fully established. Methods such as TIDAL, πR2, and prior action-chunk correction approaches already explore updating actions with fresh observations while avoiding repeated expensive backbone inference. Although the proposed denoising schedule differs, its benefits over these alternatives are not directly demonstrated.
-
The efficiency claims need a more careful interpretation. Table 2 shows that S-WAM has lower throughput than vanilla execution with the same 50-action horizon (292.7 vs. 402.5 actions/s). The reported 3.62× speedup is relative to the short-horizon baseline, rather than a reduction in inference cost at the same execution horizon. End-to-end control latency and actual responsiveness on the robot are also not sufficiently evaluated.
Questions for authors
-
Can the authors provide ablations separating staircase denoising, observation refresh, and future latent prediction? In particular, how does the method perform when only the current visual features are refreshed, without future prediction?
-
How much of the improvement comes from streaming-aligned training rather than the inference schedule itself?
-
Can the authors provide a more direct comparison with TIDAL, πR2, or lightweight online action correction methods under matched backbone and inference settings?
-
Could the authors clarify the throughput comparison and report end-to-end control latency or reaction time on the real robot?
-
Why is the improvement on DOMINO relatively modest (19.3% vs. 16.2%), with some individual tasks showing performance degradation?
6 comments
Response - 2
W2 & Q3: Comparison with TIDAL, πR², and Existing Action Chunk Correction Methods
Thank you for the suggestion. We adapted the core mechanisms of TIDAL, πR², and RNR-DP to π0.5 and evaluated them under the same single-GPU synchronous setting. S-WAM achieves the highest success rate (96.5%) and 346.8 actions/s, comparable to TIDAL (335.3 actions/s) and 2.16× and 2.29× faster than RNR-DP and πR², respectively. The three baselines process 24× more training samples than S-WAM.
1. Comparison with TIDAL, πR², and RNR-DP
All methods use the same π0.5 backbone, training data, and initialization, with an action chunk length of 50. We adapted πR² and RNR-DP from their official code and implemented TIDAL based on its paper:
- πR²: Performs one denoising step and executes 10 actions per call, requiring 5 VLM calls per 50 actions. The original method uses asynchronous VLM inference on a separate GPU; we use synchronous execution on a single GPU for a consistent comparison.
- RNR-DP: Preserves its noise relay mechanism but changes the relay interval from one action to 10 actions, with 5 noise levels and 5 VLM calls per 50 actions.
- TIDAL: Calls the VLM once per 50 actions and uses a CNN to extract motion features from pixel differences every 10 actions, followed by one-step action generation. We adapt its original setting from (H, N, K) = (16, 4, 4) to (50, 10, 5).
S-WAM uses batch 8 × 10K training steps, while vanilla uses batch 8 × 30K steps. All three adapted baselines use batch 64 × 30K steps. We measure all methods on the same L40S GPU (batch 1, bf16, eager), including remeasuring vanilla and S-WAM. Thus, the absolute speeds differ from the H100 results in the paper.
| Method (π0.5) | Spatial | Object | Goal | Long | Avg | Actions/s | TTFA (ms) |
|---|---|---|---|---|---|---|---|
| Vanilla, H_exec = 10 | 94.6 | 99.0 | 91.6 | 87.4 | 93.2 | 106.7 | 93.7 |
| Vanilla, H_exec = 50 | 66.4 | 67.6 | 72.8 | 58.6 | 66.4 | 533.7 | 93.7 |
| πR² | 93.2 | 91.8 | 89.4 | 86.0 | 90.1 | 151.4 | 62.6 |
| RNR-DP | 93.4 | 87.6 | 88.0 | 73.2 | 85.6 | 160.6 | 62.3 |
| TIDAL | 95.0 | 93.6 | 86.4 | 90.2 | 91.3 | 335.3 | 59.2 |
| S-WAM | 98.0 | 98.6 | 95.2 | 94.2 | 96.5 | 346.8 | 61.7 |
Compared with vanilla H_exec = 10, S-WAM achieves 3.25× higher throughput and 34% lower TTFA. Compared with the faster H_exec = 50 baseline, it improves success by approximately 30 percentage points.
πR² and RNR-DP use position-dependent noise schedules, which differ from the shared flow time used during π0.5 pretraining and may require more training to adapt. S-WAM keeps a shared flow time and reads out actions at different denoising stages. TIDAL also caches the VLM context, but refreshes motion features and regenerates actions in one step. In contrast, S-WAM continuously refines unexecuted actions using updated visual and predicted future features.
S-WAM and TIDAL have similar throughput because both use one VLM call per 50 actions and lightweight updates every 10 actions. πR² and RNR-DP require 5 VLM calls under our single-GPU setting, resulting in lower throughput. These results compare the adapted mechanisms under a unified π0.5 setting, rather than the full performance of the original systems.
2. Relation to Action Chunk Correction Methods
Existing methods either terminate an action chunk early and replan based on uncertainty or prediction errors, or correct generated actions using new observations. S-WAM instead refreshes the predicted future representation during generation to refine unexecuted actions. These approaches are complementary: prediction errors can trigger early replanning (Sec. 5.4), while lightweight action correction can also be applied to S-WAM outputs.
Response - 3
W3 & Q4: Throughput Comparison and Real-Robot End-to-End Latency
Thank you for the question. The reported 3.62× speedup is relative to H_exec = 10, which achieves comparable success. Compared with H_exec = 50, S-WAM requires more computation but achieves substantially higher success. We will clarify this trade-off and report additional real-robot latency measurements.
1. Throughput Comparison
| Method | LIBERO | LIBERO-Plus | Actions/s | Speedup | TTFA (ms) |
|---|---|---|---|---|---|
| Vanilla, H_exec = 10 | 95.2 | 87.2 | 80.9 | 1.00× | 123.6 |
| Vanilla, H_exec = 50 | 82.4 | 70.6 | 402.5 | 4.98× | 124.2 |
| S-WAM | 97.7 | 87.9 | 292.7 | 3.62× | 73.3 |
Compared with H_exec = 50, S-WAM uses more computation (1550 vs. 1010 GFLOPs per 50 actions) but improves success by 15.3 points on LIBERO and 17.3 points on LIBERO-Plus.
2. Real-Robot Deployment (xArm 850 + Jetson Thor)
We measure real-robot performance using an Intel RealSense D435 camera and onboard inference on Jetson Thor, with the same CUDA Graph and prompt-cache optimizations. We set the target command rate to 200 Hz and measure the achieved rate for each method; while the next sub-chunk is being computed, the robot does not execute further actions. S-WAM achieves 140.5 Hz, 1.85× that of H_exec = 10 (76.1 Hz). H_exec = 50 reaches a higher rate (157.4 Hz) because it updates only once per 50 actions, but its success rate is much lower (82.4 vs. 97.7 on LIBERO).
| Method | Command Rate (Hz) ↑ | Speedup | TTFA (ms) ↓ |
|---|---|---|---|
| Vanilla, H_exec = 10 | 76.1 | 1.00× | 89.7 |
| Vanilla, H_exec = 20 | 110.3 | 1.45× | 93.1 |
| Vanilla, H_exec = 50 | 157.4 | 2.07× | 92.7 |
| S-WAM | 140.5 | 1.85× | 75.8 |
S-WAM requires only 17.4 ms for a sub-chunk update, compared with about 90 ms for a full vanilla policy inference. Its first sub-chunk requires only one denoising step, reducing TTFA to 75.8 ms.
The observation-to-command latency is 104.3 ms at chunk start and 46.1 ms at sub-chunk boundaries for S-WAM, compared with 117.5–120.8 ms for vanilla. This measures the time from image capture to command dispatch, including image acquisition, preprocessing, and inference.
The onboard inference breakdown is shown below (separate profiling with synchronization after each module, so the totals are higher than in the end-to-end runs; ms per 50 actions):
| Component | S-WAM | H_exec = 10 | H_exec = 20 | H_exec = 50 |
|---|---|---|---|---|
| VLM | 49.6 | 262.6 | 130.9 | 51.7 |
| Visual encoding + future prediction | 56.0 | 56.2 | 27.5 | 10.8 |
| DiT denoising | 24.9 | 124.7 | 60.6 | 24.2 |
| CPU / Others | 32.0 | 102.2 | 47.9 | 18.4 |
| Total | 162.5 | 545.6 | 266.9 | 105.1 |
S-WAM requires 1 VLM call and 5 DiT forward passes per 50 actions, compared with 5 VLM calls and 25 DiT passes for H_exec = 10. Their visual encoding and future prediction costs are nearly identical (56.0 vs. 56.2 ms). Including these additional costs, S-WAM requires only 0.30× the total inference time of H_exec = 10 (162.5 vs. 545.6 ms).
Overall, S-WAM reduces TTFA through staircase readout and avoids repeated full VLM inference through lightweight sub-chunk updates, achieving a higher measured command rate than H_exec = 10 while refreshing visual feedback just as often.
Q5: Limited Improvements and Task-Level Drops on DOMINO
Thank you for the question. DOMINO is challenging, with low overall success rates. We train only on clean demonstrations and jointly train all tasks, which may contribute to performance variation across tasks. S-WAM improves some tasks but performs worse on others, such as move-can.
Unlike static settings, execution throughput is particularly important in dynamic scenes. In our experiments, frequent full replanning performs poorly: vanilla H_exec = 10 achieves only 2.2%, compared with 16.2% for full-chunk execution (H_exec = 75). S-WAM further achieves 19.3%, combining the efficiency of long action chunks with observation-based refinement of unexecuted actions. This may help maintain action quality, consistent with our findings on static tasks.
Response - 1
We thank the reviewer for the suggestion. S-WAM integrates staircase denoising, observation refresh, and streaming-aligned training: staircase denoising enables progressive sub-chunk execution, observation refresh updates unexecuted actions using new observations, and streaming-aligned training supervises intermediate readouts. We ablate future-latent prediction, observation refresh, and the training procedure while retaining staircase inference. Each is important for S-WAM's performance; removing future-latent prediction alone reduces success from 97.7 to 89.2.
1. Component ablations
Each variant modifies only the specified setting, with all other settings unchanged.
Future-latent prediction and its training provide an additional 8.5-point gain beyond refreshing current visual features alone, with the largest improvement on LIBERO-Long (95.0 vs. 78.4). Without observation refresh, success drops to 81.8, comparable to full-chunk open-loop execution (82.4). Using standard training with staircase inference further reduces success to 32.3, highlighting the need for aligned training.
Replacing only the predicted future latent at inference with one from another episode reduces success from 97.7 to 4.9, further indicating that the trained policy relies on future-latent conditioning.
2. Ablations of streaming-aligned training objectives
We further ablate individual training designs. The table reports success-rate changes (percentage points) relative to full S-WAM; training–inference alignment is examined in Q2.
Blocking action-loss gradients to the future predictor causes the largest degradation (−11.5 points), showing the importance of learning future representations useful for action generation, beyond merely predicting future features. Changing the prediction horizon also hurts performance (−4.6), while the auxiliary loss on non-executed positions has a smaller effect (−0.7).
We thank the reviewer for the question. We conduct a training–inference cross-comparison. Neither change alone achieves the full performance: streaming-aligned training with standard inference reaches 25.6, standard training with staircase inference reaches 32.3, and combining both reaches 97.7.
The first four rows use the same S-WAM architecture and future predictor, varying only training and inference. The last two use the vanilla model as references. Standard inference replans every 10 actions (Hexec=10); staircase inference invokes the VLM once per 50 actions and refreshes conditioning and emits a sub-chunk every 10 actions.
Under staircase inference, streaming-aligned training improves success from 32.3 to 97.7. Conversely, switching the streaming-trained model to standard inference reduces success to 25.6 (or 7.1 with Hexec=50). Applying staircase inference directly to vanilla also reduces success from 95.2 to 64.5.
These results demonstrate that streaming-aligned training and staircase inference are strongly interdependent, rather than providing independently additive gains. Streaming-aligned training enables intermediate action readouts and refreshed conditioning during staircase inference, rather than independently improving standard inference. We will include these ablations in the revision.