acceptodds
Withdrawn from ICLR 2027

One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models can use visual prediction to anticipate future states, but dense visual features make the generative sequence grow with the number of camera views, prediction horizon, and encoder resolution. Whether such dense representations are necessary for effective control remains unclear. We introduce OneWM-VLA, which represents each retained camera view with one predictive token per future step. Adaptive Attention Pooling compresses visual features into compact latents, which are jointly generated with robot actions under a conditional flow-matching objective. Future observations provide the latent targets during training and are not required at inference. This design incorporates visual prediction into a pretrained VLA policy while keeping the generative sequence compact. On MetaWorld MT50, OneWM-VLA improves the average success rate of the backbone from to , reaching after 60k training steps. It also achieves success on LIBERO and raises Fold Cloth success on a real Piper arm from to relative to . Comparisons on two additional VLA backbones consistently favor one token over three across the evaluated checkpoints. A matched ablation at a longer action horizon further shows that removing the latent loss reduces success from to , supporting the benefit of future supervision for policy learning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-language-action

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EdWXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OneWM-VLA, a VLA that predicts compact future visual representations for better action generation. The authors compress each future frame into a single latent token per retained camera view using Adaptive Attention Pooling. They then jointly predict future latent tokens and actions with a shared flow-matching objective, where each action token can condition on the corresponding future latent token. Future observations are used only as latent supervision during training, while the model predicts both future latents and actions at inference.

Critical strengths
  1. The paper asks a clear and practically relevant question about predictive visual bandwidth. This provides a useful design perspective for reducing the cost of future-aware policies.
  2. This paper provides an extension analysis with ablation studies by varying the number of future visual observation lengths, the target-construction mechanism, and the latent loss.
Critical weaknesses
  1. The paper does not provide a successful dense-feature baseline. Since the main claim is that a highly compressed future visual representation is sufficient for action generation, methods that explicitly use future visual prediction or latent world modeling, such as DUST [1] and FLARE [2], are more direct comparisons than standard action-only VLA baselines. Without such comparisons under a matched backbone and training setup, it is difficult to argue that the gains come from the proposed one-token representation or simply from adding future-prediction supervision.
  2. The performance gain seems to be marginal, or sometimes it performs worse than pi0. Although it sometimes outperforms pi0 significantly, the gain is not consistent (e.g., on the MetaWorld very hard task, OneWM-VLA gets 40.0% while pi0 gets 48.0%). Although the method shows substantial gains in some settings, the improvements vary considerably across task groups and action horizons. Therefore, it is difficult to conclude that OneWM-VLA provides a consistent advantage over the underlying pi0.
  3. The paper lacks conceptual novelty. Compact predictive representations and joint generation of future visual latents and robot actions have already been explored in prior work; e.g., DUST finds that 2x2 pooling the future observations shows better performance than using features without pooling.
  4. In Table 4, it only uses 1, 3, 6, and 256 tokens, which does not support whether K=1 provides meaningful performance since the paper does not report the performance when using 256 tokens (e.g., what if we use 64 tokens, which may show much higher performance?).

[1] Zheng et al., FLARE: Robot Learning with Implicit World Modeling, CoRL 2025
[2] Won et al., Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model, ICML 2026

Questions for authors

Please answer the weaknesses part.

Reviewer_PjZiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OneWM-VLA, which compresses each retained camera view into one predictive token per future step and jointly generates future visual latents and robot actions through conditional flow matching. Adaptive Attention Pooling constructs compact latent targets from future observations during training, while inference requires only current observations. Experiments on MetaWorld MT50, LIBERO, and a real Piper robot report improvements over the underlying VLA policy. Additional studies examine token budgets, pooling methods, training duration, and representation properties.

Critical strengths
  1. The paper investigates a relevant question: how much predictive visual bandwidth is necessary to support effective robotic control
  2. The evaluation includes two simulation benchmarks, real-robot experiments, and additional token-budget comparisons on two other VLA backbones.
  3. The paper provides extensive ablations and representation diagnostics, and explicitly acknowledges that one token is an effective configuration rather than a universal optimum.
Critical weaknesses

W1. The latent-loss ablation does not isolate the benefit of inference-time future generation. In Table 16, removing the latent loss reduces success from 58.09% to 21.64%, but the control retains the latent tokens, projections, and attention connections through which actions access the latent stream. This establishes the importance of supervising that stream within the proposed architecture, but does not demonstrate that joint future generation outperforms simpler alternatives. The unsupervised latent stream may itself introduce optimization difficulties. Additional controls should include an action-only model without future latent tokens, a model using future prediction solely as a training-time auxiliary objective, and a supervised latent model that prevents actions from attending to future latents. These comparisons would distinguish auxiliary representation-learning benefits from the benefits of using predicted states during inference.

W2. Representation diagnostics do not establish sufficient control-relevant future information. Effective rank, task separation, and retrieval measurements provide evidence against representational collapse, but do not directly establish whether the latents accurately encode control-relevant factors such as object positions, contact states, or motion. Global R@1 is 0.510%, above the 0.051% chance level but still low in absolute terms. This measurement alone cannot determine whether the representation preserves sufficient information for control. State-probing and intervention experiments are needed to evaluate the accuracy of the encoded future information and determine whether action predictions meaningfully depend on it.

W3. Real-world validation is limited in scope and scale. The real-robot experiments are restricted to three tasks on a single Piper platform, with 20 trials per task under each evaluation condition. These results provide useful initial evidence, but leave generalization across manipulation settings and embodiments uncertain. The evaluation does not systematically probe the limits of compression through precision-sensitive tasks, contact-state variations, or controlled disturbances during execution. Such experiments would help establish when the one-token representation preserves sufficient information and when its compression becomes a limitation.

Questions for authors

Q1, corresponding to W1. How does OneWM-VLA compare with an action-only model, a training-only future-prediction auxiliary objective, and a supervised latent model without latent-to-action attention under matched training conditions? Do the gains require generating and accessing future latents during inference?

Q2, corresponding to W2. Can the predicted latents accurately recover object positions, motion, or contact states through controlled probes? How do interventions such as replacing, shuffling, or temporally misaligning the predicted latents affect action predictions and task success?

Q3, corresponding to W3. Can the real-world evaluation be extended to tasks that systematically vary spatial precision, contact requirements, or unexpected disturbances? How does one token compare with wider representations in these settings, and what are its characteristic failure cases?

Reviewer_JVQmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether future visual observations can be represented using a single latent token per camera view and time step while retaining sufficient information for effective action prediction in vision-language-action (VLA) models.

To this end, the authors propose OneWM-VLA, which uses an adaptive attention pooling mechanism to compress visual features extracted from both current and future frames. The model jointly predicts future visual latent representations and robot actions using a conditional flow-matching objective, with future frames providing supervision for the predicted visual latents during training.

The proposed method is evaluated across multiple simulated and real-world manipulation tasks, demonstrating improvements in action prediction performance over several baselines while reducing computational overhead. Ablations investigate the contributions of adaptive pooling, predictive visual bandwidth (i.e., the number of tokens per camera view and frame), and future-latent supervision, and provide promising signals of their importance. However, more controlled ablations are needed to isolate the contributions of these components and establish the paper's core contributions.

Critical strengths

The paper addresses an important question of computational efficiency in vision-language-action (VLA) models, particularly relevant to robotic applications where low-latency, real-time action prediction is essential. The proposed architectural modifications are relatively simple, which makes the approach potentially easy to integrate into existing VLA architectures. The experiments cover diverse simulated and real-world manipulation tasks and demonstrate promising performance improvements over relevant baselines. Moreover, the findings are consistent across multiple VLA backbones, further strengthening the empirical contribution of the work.

Critical weaknesses

The contribution of this paper can be interpreted in two ways:

  1. the proposed architectural modifications enable VLAs to operate with extremely compressed future visual representations without sacrificing performance, something that existing approaches cannot achieve; or
  2. the paper empirically demonstrates that VLAs can perform effectively with only one predictive visual token per camera view and future time step.

The former would be a stronger methodological contribution; but the current experiments primarily support the latter. The paper already includes ablations of its main architectural components, but some of them lack important controls making it difficult to establish which components are responsible for the reported improvements. In particular:

  • Limited predictive bandwidth ablation: Table 4 only evaluates K ∈ {1, 3, 6}, while the full 256-token configuration runs out of memory. Without higher values of K (e.g. 32, 64, 128), it's not possible to draw conclusions about performance-efficiency trade-offs. Also, the ablation simultaneously changes the number of tokens representing current observations and future visual latents, making it difficult to isolate the effect of predictive bandwidth.

  • Incomplete ablation of joint future-latent and action prediction: The ablation in Table 16 removes the future-latent prediction loss but retains the latent prediction branch, including its tokens and attention connections to the action stream. So it doesn't cleanly isolate the impact of the joint flow matching.

  • Missing computational efficiency comparisons against baselines: Although computational efficiency is a central motivation, the paper reports latency and parameter counts only across its own predictive bandwidth configurations, which does not provide a clear picture, as the difference between the latency gap for K=6 vs K=3 and K=3 vs K=1 remains unexplained.

Questions for authors
  1. Could the authors provide an ablation that completely removes the future-latent prediction branch, rather than only removing its loss (Table 16)? This would provide a cleaner comparison and help isolate the contribution of joint future-latent and action prediction.

  2. Could the authors report inference latency, memory usage, and parameter counts for the main baselines (e.g., in Table 1)? This would help establish the practical performance–efficiency trade-off of the proposed method.

  3. Could the authors evaluate additional intermediate predictive visual bandwidth (e.g., K ∈ {16, 32, 64, 128}, where computationally feasible)?

  4. In Table 4, increasing K from 1 to 3 increases inference latency by 7.78 ms, whereas increasing K from 3 to 6 adds only 0.48 ms, despite the larger increase in sequence length. Could the authors explain this behavior and provide details on the latency measurement protocol?

Minor formatting and presentation issues:

  • The parenthetical citations are frequently rendered as textual citations.
  • Figures 1 and 2 would benefit from improved resolution and clarity to communicate the proposed method more clearly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.