acceptodds
Withdrawn from ICLR 2027

DeltaWAM: Change-Centric Visual Foresight via Delta Tokens for Efficient World-Action Model

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World-Action Models (WAMs) offer promising foresight for robotic manipulation, yet efficient and dynamics-aware future representations haven’t been fully established. Pixel-space WAMs use representations that serve for repainting entire scenes, which brings up computational overhead, spatial-temporal redundancy and waste of model capacities on task-irrelevant information. To resolve these inefficiencies, we start from an overlooked observation: consecutive frames in physical manipulation usually share substantial contexts and differ only in structured and low-dimensional ways (e.g., object displacements and end-effector motion). In essence, it is these dynamic changes, rather than sequences of entire scenes, that an action policy needs to anticipate. Grounded in this principle, we build DeltaWAM, a World-Action model that shifts the fundamental unit of future predictions to a delta token. Each delta token is a single 1D vector that encodes the changes of dense DINO features between consecutive frames, inherently capturing dynamic transitions while reducing redundancy from shared static context. Our DeltaWAM builds on DeltaWorld, a latent world model that is pretrained on large-scale videos and forecasts change-centric futures: given a history of observed delta tokens, it autoregressively predicts the future delta sequence, one token per frame. These predicted transitions, alongside the DINO features of current observations acting as the spatial anchor, are then passed to a flow-matching action expert that generates the action chunk. Our experiments show that by focusing merely on what will change in the future, delta tokens prove to be more dynamics-aware and action-forcing, providing a natural bridge from high-level semantics to low-level control. With just 256 GPU hours on 2 H100 GPUs and 0.725B total parameters, DeltaWAM achieves high success rates(92.8%) on LIBERO while also demonstrating robust generalization under procedural perturbations on LIBERO-Pro. At inference the latency is 142.1 ms latency per action chunk and peak memory is only 3.86 GB.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World-Action Models, Robotic Manipulation, Visual Foresight, Latent Dynamics, Flow Matching, Delta Tokens

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NuLTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeltaWAM, a World-Action Model that replaces full-frame or full-latent future prediction with a single delta token per future step, reusing the representation from DeltaWorld [25]. Two components are added: Gated AdaLN injection of frozen-T5 instruction embeddings into every DeltaWorld block, making the originally unconditional world model task-conditioned, and a layer-wise KV adapter that feeds the current observation's DINO patch features together with the predicted future delta tokens into a flow-matching Action DiT. The model is evaluated on the four standard LIBERO suites, on the task-perturbation dimension of LIBERO-Pro, and on a subset of LIBERO-Plus, with emphasis on its small parameter count, low latency and low peak memory.

Critical strengths
  1. The motivation is clear and matches the form of the method. Consecutive frames in manipulation do share most of their context and differ in low-dimensional structured ways, and representing foresight as transitions rather than scenes follows from that observation directly.

  2. Bringing DeltaWorld's delta-token representation into world-action modeling is a worthwhile direction. Because the representation is already transition-centric, the pipeline avoids intermediate machinery that comparable methods require, such as the latent-action distillation stage in LaWAM [10].

Critical weaknesses
  1. The ablations in Section 4.4 do not test the claimed contribution. The third contribution states that compressed delta tokens beat full state predictions, but the decoded-future-DINO variant uses the same DeltaWorld predictions merely decoded back through the frozen tokenizer, so it differs in two uncontrolled ways at once, decoder reconstruction error (which Figure 10 shows is nonzero even given ground-truth tokens) and an order-of-magnitude longer conditioning sequence, and is not full-state prediction in either case. The test-time ablation in Figure 6(b) has the same problem: zeroing the future tokens of a policy trained to consume them corrupts its input distribution, so its collapse is expected rather than evidence that foresight is necessary, and the paper's own current-DINO-only variant, trained without future conditioning, retains most of the performance.

  2. LIBERO performance is far below the field and the closest baseline is cited but never compared against. The reported average falls below every world-action model in Table 1 and below pi0. LiLa-WAM [50] is absent from all comparison tables although it is also a latent WAM built on frozen DINOv3 without embodied pretraining, and it reaches roughly 97 percent at a smaller total parameter count and lower latency, with its largest margin on LIBERO-Long; Light-WAM [31] is likewise omitted at a comparable success rate and a peak memory footprint close to this submission's, which removes the distinctiveness of the memory claim. Given a gap of several points against methods of comparable or smaller size, the submission sits off the accuracy-cost Pareto frontier and the parameter savings do not buy a usable trade-off.

  3. Reporting on both robustness benchmarks is selective and unexplained. LIBERO-Pro defines five perturbation dimensions and Section 4.3 reports one (Task); LIBERO-Plus defines seven and Appendix B reports four, omitting language, background and noise. On the official LIBERO-Pro leaderboard, baselines remain largely robust on the object and semantic dimensions, so the claim in Section 4.3 that near-zero baseline scores reveal mechanical memorization holds on only two of the five. Appendix B states that its baselines come from [15, 54], and [54] reports all seven LIBERO-Plus dimensions, so the omission is an editorial choice rather than a data limitation.

  4. The relationship between delta tokens and latent actions is never discussed, although the two are close and the relevant work is cited. UniVLA [7] learns latent actions with an IDM encoder and FDM decoder over pairs of DINOv2 feature maps under an embedding-reconstruction objective with a quantized bottleneck; DeltaTok is an encoder-decoder over pairs of DINOv3 feature maps under a feature-reconstruction objective with a continuous single-token bottleneck. UniVLA appears in Appendix B as a baseline, yet neither Section 2 nor Section 3.1 states what distinguishes the two beyond continuity and token count. Without that discussion it is unclear whether the first contribution is a new factorization or a re-parameterization of an established one.

Questions for authors
  1. Will you provide a token-matched full-state control, for instance future DINO patch features pooled or projected to the same dimensionality and token count as the delta tokens, holding sequence length and decoder involvement fixed? Without it, does the paper still claim that a change-centric inductive bias is superior rather than that compression helps?

  2. Why are LiLa-WAM [50] and Light-WAM [31] excluded from Table 1, given that both are in the bibliography, both predate the submission by months, and both exceed DeltaWAM at comparable or smaller size without embodied pretraining? If you consider the protocols incomparable, please say so explicitly.

  3. What determined the reported subsets of LIBERO-Pro (one of five dimensions) and LIBERO-Plus (four of seven)? Since [54] already supplies all seven LIBERO-Plus dimensions, can you report the omitted ones, in particular background, object and semantic, which most directly test the claim that delta tokens filter out shared static context?

  4. How does a delta token differ conceptually from a latent action as learned in UniVLA [7] or distilled in LaWAM [10], given that all are low-dimensional codes extracted from consecutive VFM feature pairs under a reconstruction objective? If the distinction is that delta tokens are autoregressively forecast as a world model rather than consumed as a policy action space, please state that and position the first contribution accordingly.

Reviewer_4tQmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DeltaWAM, a world-action model that uses the change between frames as the unit of future prediction. The method builds on DeltaWorld, which encodes the difference between the DINOv3 features of consecutive frames into a single delta token and uses Best-of-Many training to generate multiple futures in a single forward pass. The authors add language conditioning and dual-view adaptation to DeltaWorld and first fine-tune it on LIBERO demonstrations; they then freeze it and train a flow-matching action generator that produces action chunks conditioned on the DINO features of the current frame and the predicted future delta tokens. The paper reports an average success rate of 92.8% on LIBERO with a relatively small number of parameters, low inference memory and low latency. It also reports results under LIBERO-Pro task perturbations and on LIBERO-Plus, ablations on the conditioning inputs, and a qualitative real-robot pick-and-place demonstration.

Critical strengths
  1. Complete and lightweight engineering. The paper integrates a pretrained compact video world model into action generation and obtains a working policy. Its total parameter count, inference memory, latency and training cost are clearly lower than those of most world-action models in the comparison, which has practical value in resource-constrained settings.

  2. Some results are informative. The ablations show that providing the policy with predicted futures yields a clear improvement over using only the current observation, and under LIBERO-Pro task perturbations the method outperforms the reported baselines on some suites.

Critical weaknesses
  1. Limited novelty; the work is an incremental combination. The main contributions claimed by the paper are already substantially present in prior work:
  • Predicting the future in the feature space of a vision foundation model, so that model capacity is not spent on task-irrelevant pixel details, is the central claim of DINO-world ("Back to the Features: DINO as a Foundation for Video World Models", Baldassarre et al., 2025, arXiv:2507.19468). The same work also proposes pretraining a world model on large-scale unlabeled videos and then introducing new conditioning signals to adapt it to downstream tasks, and its conclusion lists language conditioning as a future direction.
  • The observation in the first contribution, that consecutive frames differ only in structured, low-dimensional ways so that only the changes need to be predicted, as well as the design of representing the change between frames with a single delta token and reconstructing the current frame from the previous frame's features plus this change, comes from DeltaWorld ("A Frame is Worth One Token", Kerssies et al., 2026, arXiv:2604.04913).
  • The efficiency advantage in the second contribution mainly comes from the single-token representation and single-pass generation already present in DeltaWorld.
  • The conclusion of the third contribution, that compact delta tokens are sufficient to replace full states, has already been verified by DeltaWorld on dense forecasting tasks, including the finding that replacing full features with delta tokens in other world models yields nearly the same accuracy. This paper extends this conclusion to action generation.

The overall framework, in which a world model predicts the future and a DiT then generates actions from the prediction, follows the standard world-action model pipeline. What the paper actually adds is mainly the integration of this representation into action generation, together with language conditioning. The paper cites DeltaWorld, but its statement of contributions does not clearly separate what is inherited from prior work from what is new, which may lead readers to overestimate the increment. I also suggest adding a discussion of DINO-world.

  1. The combination does not lead to deeper insight. The main conclusions of the experiments, namely that providing the policy with future predictions is better than using only the current observation, that language conditioning makes the predictions more task-relevant, and that removing at inference an input the policy relied on during training causes failure, are either expected or natural extensions of conclusions from prior work. The paper does not further analyze why the change-centric representation is better suited to control, for example what action-relevant information it encodes or under what conditions it fails. The paper therefore reads more as an engineering contribution than as an advance in method or understanding.

  2. The results are not strong enough to compensate for the limited novelty. On standard LIBERO, the method falls below most baselines in the comparison, including world-action models without embodied pretraining. The improvement on LIBERO-Pro is mainly concentrated in a single suite, and the success rates on the other suites remain low. Efficiency is the main advantage of the paper, but efficiency alone is not sufficient to support the current claims of contribution.

Questions for authors

Questions that could affect my assessment:

  1. Compared with DeltaWorld and DINO-world, what do the authors consider the most important new contribution of this paper? Are there phenomena that do not appear in prior work and are observed only after the representation is connected to control?

Clarifications and suggestions (not affecting my rating):

  1. Figure 5 visualizes only the delta tokens predicted by DeltaWAM under LIBERO-Pro task variations. It would be helpful to show the same visualization for other representations on the same tasks, such as the current-frame DINO features or the decoded future DINO features, so that readers can judge whether the observed clustering is specific to the delta representation.
Reviewer_wSiiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeltaWAM, which uses compact delta tokens to predict future visual changes for robot action generation. The method combines predicted delta tokens with current DINO features and introduces language conditioning to guide future predictions. Experiments on LIBERO, LIBERO-Pro, and LIBERO-Plus demonstrate promising efficiency and robustness.

Critical strengths

1.Interesting Design: Using compact visual change representations instead of predicting complete future states provides an interesting approach to building efficient World-Action Models. 2.Good Efficiency: The model achieves 92.8% average success on LIBERO with only 0.725B parameters and 142.1 ms inference latency per action chunk. 3.Extensive Experiments: The paper includes performance comparisons, representation ablations, and robustness evaluations across multiple benchmarks.

Critical weaknesses

1.Limited Novelty and Representation Analysis: The delta token representation and its prediction framework are largely inherited from DeltaWorld. In Figure 6(a), the compared variants use substantially different numbers of visual tokens, making it difficult to determine whether the improvements come from modeling visual changes or simply using a more compact representation. 2.Insufficient Ablation of Language Conditioning: Language conditioning is an important extension of DeltaWorld, but Figure 3 mainly provides qualitative examples. Its contribution to task performance has not been directly evaluated. 3.Limited Performance and Generalization: Although DeltaWAM is efficient, its success rate on standard LIBERO remains below several competitive WAMs. The improvements on LIBERO-Pro are also mainly concentrated in Spatial tasks, while performance on other task categories remains relatively low. Further analysis would help clarify the generalization benefits of delta tokens.

Questions for authors

1.Representation Ablation: Could the authors add a baseline that compresses future DINO features to the same token budget as delta tokens? Comparing success rates under the same training settings would help clarify the benefit of change-based representations. 2.Language Conditioning Ablation: Could the authors remove language conditioning from DeltaWorld while keeping the language input to Action DiT unchanged? Please compare the success rates on LIBERO and LIBERO-Pro to evaluate its contribution. 3.Generalization Analysis: Could the authors extend the representation ablations in Figure 6(a) to selected LIBERO-Pro tasks, particularly Spatial and Goal? This would help determine whether delta tokens improve generalization under task perturbations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.