acceptodds
Withdrawn from ICLR 2027

ElasticWAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World Action Models (WAMs) benefit from imagining the future, but must they refine every region and complete future-video denoising to generate effective actions? We argue that future refinement should be guided by its contribution to action generation rather than visual completeness. We introduce ElasticWAM, an action-guided framework for adaptive spatiotemporal imagination. Built on a frozen pretrained WAM, ElasticWAM learns a unified refinement-demand field from action-prediction supervision to determine which future-video tokens to refresh, how far to advance denoising, and when to stop world refinement. It selectively updates future-video representations while retaining cached context from unrefreshed regions through the pretrained action-conditioning interface. Experiments across four manipulation benchmarks demonstrate the effectiveness of this approach. On RoboCasa-GR1, ElasticWAM improves success by 2.4 percentage points while reducing inference latency by 32.2% relative to Joint-WAM. Real-world dual-arm experiments further demonstrate long-horizon task execution under nominal conditions, lighting perturbations, and object replacement. These results support maintaining useful future context for control through selective refinement and cache reuse, without refreshing every region at every denoising step.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Robot Manipulation, Embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BVCUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ElasticWAM, a controller for allocating future-video computation in a frozen world-action model. An action-supervised refinement-demand field determines which video tokens to refresh, how far to advance denoising, and when to commit the world representation and finish action generation. Unrefreshed tokens retain cached representations, allowing active queries to access the full logical context. The evaluation covers LIBERO, LIBERO-Plus, LIBERO-Pro, RoboCasa-GR1, and a four-stage real-robot tea-preparation task. The main reported efficiency result is a 32.2% latency reduction relative to Joint-WAM on RoboCasa-GR1, with success increasing from 55.0% to 57.4%.

Critical strengths
  1. The treatment of sparse refinement is well motivated. The method separates the decision to recompute a region from the decision to retain it as conditioning context. Keeping inactive regions available through the cache is a useful design choice for manipulation, where a region that does not need another update may still matter to the action. Training the allocation decisions using action errors also gives the controller a task-relevant objective.

  2. The ablations provide meaningful evidence beyond an overall comparison with dense inference. On RoboCasa-GR1, learned selection at the same 96-token budget reaches 57.42%, compared with 53.83% for random selection. The First-KV comparison also tests the value of refreshing cached context. On LIBERO-Plus, adding adaptive progression and commitment improves over the spatial-only variant. Together with the measured latency reduction, these results make selective refinement a promising direction rather than merely a reduction in nominal token count.

Critical weaknesses
  1. The controller's training procedure is not specified well enough to reproduce or fully assess the central contribution. Section 3.3 describes converting counterfactual errors into a soft target for the Beta progression, but does not define this conversion or the temperature T(s). The soft mask used in Eq. (12) and several losses in Eq. (14) are also left unspecified. These choices determine how the proposed inference policy is actually learned; they are more consequential than ordinary implementation details. Appendix A mostly describes evaluation and does not fill these gaps or provide the controller architecture and training hyperparameters.

  2. The evidence for the headline gains needs clearer experimental accounting. Appendix A reports ElasticWAM on a 1,506-episode LIBERO-Plus subset while quoting baseline results from Faster-WAM. It is not clear whether the quoted baselines use the same subset and category weights, so the 73.7% versus 66.3% comparison is difficult to interpret confidently. The latency table also lacks the hardware and timing protocol needed to assess the 32.2% saving. Finally, the real-robot evaluation does not state the number of trials, and the reported success-rate differences have no uncertainty estimates. The favorable trends are encouraging, but these omissions limit how strongly they support the conclusions. I currently lean toward weak rejection, principally because the learning procedure and the main comparisons need this additional evidence.

Questions for authors
  1. Could the authors provide the exact Beta-target construction, T(s), soft-mask relaxation, and losses in Eq. (14), together with the controller architecture, training settings, and the frozen WAM checkpoint used? These details would materially affect my assessment of reproducibility.

  2. How was the 1,506-episode LIBERO-Plus subset selected, and what are the episode counts per category? Were the quoted Joint-WAM and Fast-WAM results evaluated on exactly this subset with the same aggregation? If not, a comparison on matched episodes would help establish the robustness gain.

  3. What hardware and timing procedure produced Table 4, and does the reported latency include controller evaluation, mask selection, and cache management under the same conditions for all methods? Please also give the real-robot trial counts and uncertainty estimates for the main success-rate comparisons. This would clarify the practical size and reliability of the improvements.

Reviewer_Sx6kOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the cost of repeatedly updating future video representations during WAM inference. The authors add a controller to a frozen pretrained model to decide when to stop world updates, how far to advance if updates continue, and which future regions to refresh at each step. On RoboCasa-GR1, ElasticWAM achieves a success rate of 57.4% with a latency of 388.2 ms. The corresponding results are 55.0% and 572.6 ms for Joint-WAM, and 54.9% and 331.1 ms for Fast-WAM. The success rate on LIBERO-Plus is 73.7%.

Critical strengths
  1. I find it reasonable to use action quality to decide how much the future representation needs to be updated. On RoboCasa-GR1, ElasticWAM improves the success rate by about 2.4 percentage points over Joint-WAM while reducing latency by 32.2%. On LIBERO-Plus, the improvement is about 7.4 percentage points. The paper also reports results for Fast-WAM, showing the trade-off between the higher success rate and the additional latency of ElasticWAM.
  2. Some of the experimental results are useful. With the same budget of 96 refreshed tokens, learned selection achieves 57.42%, compared with 53.83% for random selection. Reusing only the cache from the first round achieves 54.50%. These results show that both the choice of regions to update and whether to continue updating affect control performance.
Critical weaknesses
  1. The paper lacks sufficient experimental comparisons, analysis, and implementation details. In particular, the authors cite experimental results from Faster-WAM but do not compare their method with Faster-WAM itself. Faster-WAM also studies preserving future representations and reusing caches, so it remains unclear how much the added spatial selection and temporal control improve on Faster-WAM. As presented, the contribution appears to be an incremental extension of Faster-WAM. The authors also state that the pretrained WAM is frozen during controller training, but do not adequately explain which base model version and checkpoint are used in each experiment or whether the base model has been adapted to the task. This information directly affects the fairness of the comparisons and the interpretation of the performance gains.
  2. The overall quality of the figures and writing is poor. Figure 2, the main method figure, contains many module names and mathematical symbols, but does not clearly distinguish the existing frozen model from the new trainable components. It also fails to clearly connect the initial full update, the later sparse refreshes, cache reuse for regions that are not refreshed, and action denoising after world updates stop. As a result, it is difficult to understand from this figure alone how the unified spatiotemporal refinement-demand field coordinates the three decisions or how it differs from existing scheduling and caching methods.
Questions for authors
  1. Please add a comparison with Faster-WAM or a suitable control variant, and explain any differences in the backbone, data, training cost, and timing conditions. Are there results under the same budget for independent temporal scheduling combined with spatial scoring? These results would directly affect my assessment of the new contribution.
  2. Please specify the controller inputs and architecture, the two error terms in Eq. (10), the temperature in Eq. (11), and the definitions of the losses in Eq. (14). How are the candidate step sizes, soft masks, and safety stopping conditions set? Which input configurations correspond to 196 and 128 tokens, respectively?
  3. How are the velocity estimates and layer-wise caches of regions that are not refreshed used in later updates at different noise levels? When inference directly uses the location parameter of the temporal distribution, what role does its shape parameter play? How are differences between sparse iteration states during training and deployment handled?
  4. How is the overall architecture designed, and what is the reasoning behind it? Figure 2 needs to be redrawn.
Reviewer_Zu6COpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ElasticWAM, a controller trained on a frozen pre-trained world action model (WAM). At inference time it determines when to stop refining the imagined future video, how far to advance the noise level at each refinement step, and which future-video tokens to refresh. ElasticWAM does a required dense joint video and action update, refreshing a fixed budget of K=96 future-video tokens each step, and reuses the cached hidden states and KV of the other tokens. It continues until a cumulative “World Commit” criterion fires, at which point the action branch finishes denoising.

Critical strengths
  1. The question is well asked and practically relevant. Is a dense and complete denoising of the imagined future really necessary for WAM-based control? A reasonable course is to allocate refinement according to its utility for the action rather than by visual completeness. A practical design is to refresh only some queries and keep the full KV context as a plug-in on a frozen WAM. Some results confirm the basic intuition. Refreshing 96 tokens is better than refreshing all 196 on the LIBERO-Plus subset (73.71 vs. 70.92, Table 6b). The learned selection outperforms the random selection with the same budget on RoboCasa-GR1 (57.42 vs. 53.83, Table 5). If so, this is an interesting and informative result that over-refining the imagined future can hurt control.
  2. The evaluation is extensive. It contains four simulation benchmarks and a long-horizon real-robot task under object and lighting OOD. The progress score usefully supplements full-task success.
Critical weaknesses
  1. The main empirical claims rest on non like-for-like comparisons, and the gains are largely in the noise. (a) LIBERO-Plus is not evaluated on the same episodes as the baselines. The paper evaluated ElasticWAM on a subset of LIBERO-Plus consisting of 1,506 episodes (1,110 successes, 73.71%) as shown in Appendix A, and all baseline numbers in Table 3 are from the full-benchmark evaluation in Faster-WAM (Zhao et al., 2026). So the statement in Sec. 4.2 that ElasticWAM improves over Joint-WAM "across all seven perturbation categories" compares numbers computed on different episode sets. The paper doesn’t say how the subset was selected either. The subset used in the ablations in Table 6.
  2. No variance estimates, and most gains are small relative to noise. It’s also not the current SOTA. Supervised training introduces some additional parameters. Whether the performance improvement resulting from this is related to these parameters remains to be further verified.
Questions for authors

1.Could you give us the full-benchmark LIBERO-Plus results, or re-run Fast-WAM and Joint-WAM on the same subset of 1,506 episodes? How was the subset selected?

2.Which frozen pretrained WAM checkpoint is employed? How many demos per task were used in RoboCasa-GR1? How were the three baselines trained? For LIBERO-Pro, re-run the number of episodes and the baseline numbers that were quoted?

  1. What is the success rate and latency of Joint-WAM with 2, 3 and 5 fixed denoising steps on RoboCasa-GR1, ideally plotted as a curve together with Fast-WAM and ElasticWAM? What hardware is used? What is the mean and worst case latency per replan?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.