- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AnyStep-WAM, a framework for adaptively selecting the computational budget during the deployment of world-action models. The key methodological contributions are distilling a frozen teacher WAM into an interval-conditioned student and training a lightweight model that uses teacher flow path curvature to predict difficulty and required integration fidelity during deployment. The method is evaluated in simulation on RoboTwin 2.0 and in the real world on six bimanual manipulation tasks. In these experiments, inference time decreases by about 40-64% while maintaining success rates comparable to the base policy.
Critical strengths
The paper addresses a highly relevant problem: reducing the high computational burden of world-action models. The proposed method requires extensive additional training, but the experiments show that it can substantially reduce the computational burden at inference time.
Critical weaknesses
- The paper does not report any information about the statistical significance of the results. All experiments seem to be conducted with a single seed, and the relative difference in success rates among the evaluated methods is mostly within 0 to 2 percentage points. The lack of statistical tests across multiple seeds makes it difficult to assess whether the proposed method, as claimed by the authors, indeed maintains high success rates while reducing computation time.
- The authors write the following in the motivation of their method: "critical actions require precision, while less sensitive actions allow faster generation with fewer denoising steps". However, the paper is lacking a convincing analysis of whether AnyStep actually allocates more compute to high-precision interactions. At the top of Figure 3, predicted curvature risk is high for inference index 1, although this timestep corresponds to a free-space motion that does not require high precision.
Questions for authors
Clarification requests:
- Could the authors analyze the statistical significance of their results? For example, statistical tests across multiple training seeds would be highly valuable to assess the impact of AnyStep-WAM on the success rate.
- Could the authors analyze in what types of situations their method reduces computational effort the most? In Figures 4 and 5, the relationship between task phase/motion and computational budget appears somewhat arbitrary, and it is unclear whether the computational budget is actually highest during high-precision phases.
- What is the reason for choosing very different percentile values for video (10/20) and action (20/80) similarity? The ablation only evaluates +5/-5 changes to these percentile values, and it would be helpful to analyze the sensitivity with respect to these values more broadly.
- How were the weighting factors of the four loss terms chosen, and how sensitive is the overall performance of AnyStep-WAM to these values?
- Could the authors provide details about the training time of their method?
Suggestion for future work:
- Section 4, in particular Section 4.2, is very dense and sometimes hard to follow. Adding more explanations, especially visualizations, would greatly improve readability here.
- Adding one more benchmark environment, such as Libero-Plus or Robocasa, would be highly valuable to understand whether the reported behavior also transfers to other robot embodiments, such as single-arm or mobile manipulators.
13 comments
Response to W2 and Q2: Which phases receive more computation, and the high-risk free-space example
W2 and Q2: Which phases receive more computation, and the high-risk free-space example
Thank you for raising this question. We have added the following content to Appendix E.5, “Task- and Phase-Level Computation Allocation,” in the revised paper. Taking Motus as an example, Table 26 in Appendix K reports results from 100 trials per task per setting. Broad-target transport and placement tasks use fewer denoising steps: Place Empty Cup, Place Container Plate, and Place Burger Fries use only 2.74–2.78 steps on average. Coordinated or geometrically constrained tasks receive larger budgets: Handover Block uses 5.42 steps, Put Object Cabinet uses 5.99, and Hanging Mug uses 6.45, compared with the 50-task average of 3.98.
| Task | Representative requirement | Clean mean | Randomized mean | Average |
|---|---|---|---|---|
| Place Empty Cup | Broad-target transport and placement | 2.94 | 2.54 | 2.74 |
| Place Container Plate | Placement into a relatively tolerant target region | 2.86 | 2.70 | 2.78 |
| Place Burger Fries | Short transport and broad-region placement | 2.71 | 2.80 | 2.76 |
| Blocks Ranking RGB | Color-based semantic ordering with straightforward pick-and-place motions | 2.82 | 3.11 | 2.97 |
| Handover Block | Coordinated dual-arm transfer and release | 5.87 | 4.97 | 5.42 |
| Put Object Cabinet | Constrained placement inside a cabinet | 6.31 | 5.67 | 5.99 |
| Hanging Mug | Handle grasp and constrained hook placement | 6.46 | 6.44 | 6.45 |
| All 50 tasks | — | 4.02 | 3.94 | 3.98 |
We additionally analyze phase-level statistics from adaptive rollouts of AnyStep-Motus on Hanging Mug, including the risk score and the executed denoising budget. Free-space motion receives the smallest mean budget (4.4 steps), while the grasp/contact transition receives the largest (7.8 steps). The proportion of inference calls using rises from 34% during free-space motion to 84% during the grasp/contact transition.
| Hanging Mug phase | Mean risk | Mean | Share with | Share with |
|---|---|---|---|---|
| Free-space motion | 0.47 | 4.4 | 39% | 34% |
| Pre-contact alignment | 0.61 | 6.2 | 18% | 63% |
| Grasp/contact transition | 0.72 | 7.8 | 8% | 84% |
| Contact manipulation/placement | 0.67 | 7.5 | 10% | 80% |
We agree that a single example in Figure 3 does not provide sufficient evidence for the overall relationship between manipulation phases and computation allocation. We apologize for the limited analysis in the original paper and added the task-level and phase-level statistics above to better support and clarify this trend in the revised paper.
Response to Q3: Video/action percentile thresholds and wider sensitivity, and Q4: Selection and sensitivity of the loss weights
Q3: Video/action percentile thresholds and wider sensitivity
Thank you for suggesting a broader sensitivity analysis. We have added the following results to Appendix E.6, “Video/Action Percentile Thresholds and Sensitivity,” in the revised paper. We compute the easy and hard similarity thresholds from the specified percentiles of each modality’s offline training-set distribution. Predicted risk interpolates between these two threshold values, and the scheduler selects the smallest budget satisfying both video and action fidelity requirements. If no budget qualifies, it selects the budget with the largest summed predicted fidelity.
The asymmetric defaults reflect the different roles of the two modalities. Video similarity captures agreement in predicted visual dynamics, including appearance and background changes that may be less relevant to control precision. Its 10th/20th percentile thresholds therefore provide a relatively permissive scene-consistency check. Action similarity measures agreement in the commands executed by the robot. The 20th/80th percentile thresholds allow a permissive requirement in easy states and require higher action fidelity in high-risk states.
We have expanded the Motus sensitivity evaluation as shown below, keeping the student and scheduler fixed and changing only the inference thresholds. SR and mean steps are averaged across clean and randomized settings.
| Configuration (Motus) | Video percentiles (easy/hard) | Action percentiles (easy/hard) | SR (%) | Mean steps |
|---|---|---|---|---|
| Default | 10 / 20 | 20 / 80 | 87.96 | 3.98 |
| Joint −5 | 5 / 15 | 15 / 75 | 87.60 | 3.08 |
| Joint +5 | 15 / 25 | 25 / 85 | 87.93 | 5.85 |
| Video only: lower | 5 / 10 | 20 / 80 | 87.81 | 3.54 |
| Video only: higher | 20 / 40 | 20 / 80 | 87.86 | 4.67 |
| Action only: lower | 10 / 20 | 10 / 40 | 87.02 | 2.61 |
| Action only: lower hard threshold | 10 / 20 | 20 / 50 | 87.47 | 3.25 |
| Action only: higher | 10 / 20 | 30 / 90 | 87.89 | 5.16 |
| Shared 10 / 20 | 10 / 20 | 10 / 20 | 86.74 | 2.23 |
| Shared 20 / 80 | 20 / 80 | 20 / 80 | 87.62 | 6.71 |
SR is relatively stable over most of the tested range: 8 of the 10 configurations remain within 0.49 percentage points of the default, while mean steps vary from 2.23 to 6.71. With action percentiles fixed at 20/80, varying the video percentiles changes SR by at most 0.34 points. More permissive action settings of 10/40 and 10/20 reduce SR by 0.94 and 1.22 points, respectively. Higher thresholds increase computation without improving SR over the default. These results support the default as a practical SR–computation trade-off, while showing that SR is less sensitive than the computational budget across most tested settings.
Q4: Selection and sensitivity of the loss weights
Thank you for asking about the loss weights. We have added the following results in Appendix C.2 of the revised paper. We use based on the roles of the four terms. The endpoint loss receives the largest weight because the one-step endpoint is both an important inference mode and the preview reused by larger budgets. The interval loss provides direct supervision for budget-aligned finite-time transitions. The flow-matching term maintains instantaneous-velocity prediction, while the recovery term supervises continuation from preview-recycled student states. These weights were fixed across Motus, FastWAM, and LingBotVA and were not tuned on the test set.
To evaluate sensitivity on Motus, we test two alternative combinations. Alternative A assigns more weight to endpoint and interval supervision, whereas Alternative B assigns more weight to flow matching and recovery. For each setting, we retrain both the AnyStep student and its scheduler under the same protocol.
| Configuration (Motus) | Fixed 1-step SR (%) | Fixed 4-step SR (%) | Fixed 10-step SR (%) | Adaptive SR (%) | Adaptive mean steps | |
|---|---|---|---|---|---|---|
| Default | (0.20, 0.50, 0.30, 0.20) | 82.12 | 87.04 | 88.09 | 87.96 | 3.98 |
| Alternative A | (0.10, 0.60, 0.40, 0.10) | 82.38 | 87.15 | 87.92 | 87.78 | 3.86 |
| Alternative B | (0.30, 0.40, 0.20, 0.30) | 81.76 | 86.81 | 88.16 | 87.84 | 4.19 |
Performance remains stable across the tested configurations: fixed-budget SR differs from the default by at most 0.36 percentage points, and adaptive SR by at most 0.18 points, with mean budgets of 3.86–4.19 steps. Alternative A has slightly higher one- and four-step SR and a lower mean adaptive budget, while Alternative B has slightly higher ten-step SR and a higher mean adaptive budget. The default achieves the highest observed adaptive SR among the three configurations. These results suggest limited sensitivity to the two tested weight redistributions.
Response to W1 and Q1: Statistical significance and robustness across training seeds
Dear Reviewer WuyW,
Thank you for your careful review and constructive feedback. Due to the 5,000-character limit per response, we have divided our replies into multiple comments.
W1 and Q1: Statistical significance and robustness across training seeds
Thank you for highlighting the need for statistical validation. We have added the following results to Appendix H, “Robustness across Training Seeds,” in the revised paper. In addition to the original training seed 42, we trained the student and scheduler with seeds 43–46 for all three backbones. Each evaluation uses 100 trials per task per setting, with SR averaged across tasks and clean/randomized settings. The table reports mean ± sample SD across the five training seeds.
Across the five training seeds, adaptive inference achieves SRs of 88.03 ± 0.06%, 91.66 ± 0.07%, and 92.26 ± 0.04%, with mean denoising steps of 3.96, 4.99, and 3.63 on Motus, FastWAM, and LingBotVA, respectively. The small SDs indicate consistent performance across training seeds. Adaptive SR remains close to that of the fixed 10-step student while using substantially fewer denoising steps.