- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper presents WorldToken, a new robot learning policy architecture emphasizing on history information processing where a causal transformer encodes history information into a few latent tokens (dubbed world token in the paper) and DiT decodes the latent tokens into continuous action through iterative denoising. Simulation experiments ablated various model hyperparameter choices.
Critical strengths
- Timely topic. Memory for robot learning is becoming more and more important as a way of improving upon the mainstream current-state-only policies.
Critical weaknesses
-
Certain architecture design choices are ill-motivated and not ablated. It is fairly standard to use an encoder-decoder architecture for encoding history information, however conditioning the DiT action decoder solely on one world token and expect them to encode the full current and history visual, proprioception, and task command information is ill-motivated and unabated (e.g. against a design where the action decoder also conditions on full current state information). Further, since the training supervision completely comes from the action diffusion objective, nothing in the architecture design explicitly prevents the causal transformer encoder from learning a degenerated world tokens (say that simply copies or averages past history). This concern is also not addressed in ablation studies. Finally, with the default 10-step context length and the ~0.2s difference between each context step used in most of the experiments from the paper, the model can encode ~2s of history, and whether such a short history length provides enough benefits is unclear. Although the paper shows one long context task (RMBench Blocks Ranking), the authors fail to establish consistent benefits from their method through evaluation on multiple tasks, and the RMBench Blocks Ranking training loss is manually engineered as discussed in Appendix F.3.
-
Missing baselines and incomplete comparison. Given that there is plenty of memory for robot learning works (and policies without explicit memory design), it is unusual that the paper’s experiments compare the authors method against a very limited set of external baselines, making it hard to gauge the contribution of the paper’s architecture design. In addition, the paper does not provide real world experiments to validate the results in simulation. Finally, some experiment design is questionable in rigor. For example, in the RMBench Blocks Ranking experiment at Table 5 the authors train the model with a long context length but evaluate the model with different context length, whereas the previous experiments in Table 3 show that mismatch between training and inference time context history length can drastically hurt performance.
Questions for authors
-
Can the authors provide additional justification or ablations on various architectural designs including design to condition the action decoder on only the world tokens?
-
Can the authors explain whether or why the causal transformer encoder does or does not learn a degenerated mapping between history information to the world tokens through additional experiments?
-
How does the WorldToken policies compare to other robot learning baselines with or without explicit memory design?
3 comments
Acknowledgement of Rebuttal
Thank you for providing a rebuttal that clarifies the methodology and provides useful context. However, since my main concerns about limited comparison with existing temporal model architectures and lack of compelling evidence for the methodology's advantage remain, I would like to maintain my original rating.
Response to Reviewer vhPn
To address the remaining concern, we would like to clarify two points: a comparison with externally developed temporal systems and a controlled evaluation of our proposed temporal interface.
Regarding why we did not compare with other temporal systems, we do not claim exhaustive superiority over the rapidly emerging family of dedicated memory systems, since our study does not focus on that area. Rather, long-context scalability is an architectural capability enabled by WorldToken’s one-token-per-timestep interface. Therefore, our experiments are designed to demonstrate that the proposed method has great potential in long-context settings. Meanwhile, we agree that adding comparison experiments with recently developed temporal systems would better demonstrate WorldToken’s potential. We therefore plan to conduct experiments using Mamba for full-history recurrent compression and add the results to the appendix.
The claimed architectural advantage of WorldToken is specifically its compact one-token-per-timestep interface. We support this claim in three ways. First, WorldToken processes a 608-step, approximately 145-second context without a separate memory module. Second, fixed-checkpoint history interventions reduce success from 95% to 28% when the visible history is shortened from 146 to 8 seconds, confirming that the policy actually uses distant information. Third, retaining 50 tokens per timestep increases temporal-backbone FLOPs by a factor of 55.31 while providing only a 2.09-point improvement in success at D300. At D2900, the single-token model matches or exceeds the 50-token variant in terms of success while achieving lower RMSE. These results demonstrate a favorable trade-off among per-timestep spatial detail, temporal coverage, and computation under the evaluated settings.
Beyond these empirical findings, we further clarify the broader architectural motivation behind WorldToken’s one-token-per-timestep design.
We rethink history modeling at the architectural level because we believe memory is even more fundamental to robotics than commonly assumed—much as in daily life, a blind person can still live independently and accomplish many complex tasks, whereas someone with severe memory impairment struggles to complete even basic multi-step actions. Looking back at the history of NLP, mainstream recurrent memory mechanisms must progressively compress history into a fixed state, which creates inherent bottlenecks for long-horizon modeling and scaling. As a result, it was ultimately superseded by Transformers. We therefore adopt a structural paradigm similar to LLMs, aiming to replicate the methodological evolution and scaling trajectory of LLMs in robotics. Along this direction, WorldToken makes a deliberate trade-off between temporal and spatial information: allocating more representational and computational budget to extending the temporal sequence rather than preserving fine spatial granularity within each timestep. Our experiments demonstrate the feasibility of this trade-off: comparing N3 (K=1) in Table 1 with N2 (K=50) in Table2 shows that at D2900, the single-token N3 matches or surpasses the N2 (K=50) in success rate while achieving lower RMSE. This indicates that the key to performance is not retaining more raw information per timestep, but using a stronger encoder (and model capacity) to compress each timestep effectively. We conjecture that this also holds for more complex observations. On one hand, images possess low-dimensional structure. On the other, a closed-loop policy, much like the human eye, can retain only focal details at the current moment and adjust its gaze on demand in the next timestep to inspect other locations.
Going one step further, this one-token-per-timestep structure aligns robot sequence modeling directly with language. Just as humans can speak with their eyes closed, a robot model should have the same capability. Due to current training data scale limitations, our paper passes a CLIP text vector at each timestep, but our underlying design intent is to pass one text token per timestep. Under that formulation, masking out the other sensory inputs at a given timestep naturally reduces the model to a language model, opening up possibilities for multi-source data training and multimodal capability fusion in the future.
Response to Reviewer vhPn
Thank you for your comments. WorldToken is a compact, time-first VLA architecture rather than a dedicated memory module. We clarify its information flow, design choices, and evaluation scope below.
Our contribution is the time-first VLA architecture rather than the encoder–decoder structure itself. The encoder fuses the current images, proprioception, and task condition into one world token. A causal Transformer contextualizes the token sequence, and the DiT uses the resulting history representation. Because the encoder receives no past observations, past timesteps remain individually represented rather than compressed into one token.
About the architectural alternatives, our paper reports three diagnostics. A current-observation bypass lets every DiT block access all 50 fused current-observation tokens, increasing RMSE and reducing success from 48.12% to 46.20% and from 48.38% to 42.00%. Passing 50 rather than one token per timestep improves success by only 2.09 points and RMSE by 3.3% at D300, while requiring 55.31 times as many temporal-backbone FLOPs. Moving history processing to denoiser cross-attention yields mixed results. WorldToken has 6.4%–8.7% lower RMSE, while the DP-like variants have 1.4–3.0 points higher success.
Regarding the scope of our long-history evidence. A practical advantage of WorldToken is that each timestep adds only one temporal token, keeping long histories compact. This allows the policy to use a 608-step, approximately 146-second context on Blocks Ranking without a separate memory module. The interventions detailed in Question 2 show that this history is behaviorally useful. We present Blocks Ranking as a single-task long-history case study.
The Blocks Ranking objective affects only downward button execution and uses no history supervision. Under the standard objective, the policy reaches the correct arrangement in 99/100 episodes but has only 13 official successes because most executions miss the button threshold, obscuring long-context evaluation. Although different code revisions prevent a clean loss-only interpretation, all history interventions fix the modified checkpoint and vary only visible history. Reducing history from approximately 146 to 8 seconds lowers official/strict success from 95%/94% to 28%/24%, demonstrating long-context utilization.
Our comparisons address two questions: whether WorldToken provides effective control relative to a same size RoboCasa baseline, and whether WorldToken can reach success rates comparable to substantially larger pretrained VLAs. We therefore select BC-Transformer as the official, reproducible, from-scratch baseline and use pi0.5 as a large pretrained reference. On the same 23 tasks, D300 demonstrations, and 1,150 episodes, WorldToken achieves 46.83% mean success versus 31.28% for BC-Transformer. At D2900, the 85.3M-parameter WorldToken reaches 59.45%, close to 62.1% for the approximately 3.35B-parameter pretrained pi0.5. Published 23-task means are 51.11% for GR00T N1 and 64.96% for GR00T N1.5. Also, The K-token diagnostics described above provide controlled internal comparisons. Dedicated-memory results use different backbones, data, and protocols, so we do not treat them as same-protocol controls or claim superiority.
Our simulation experiments directly establish WorldToken’s effectiveness and its data-scaling, token-interface, and history-utilization trends under controlled conditions. These are the architectural questions studied in this paper. Sim-to-real robustness is a separate deployment question. We view real-robot experiments as an important direction for future work.
Table 5 tests whether a fixed policy uses information beyond shorter history windows, so the checkpoint must remain fixed. Training separate policies with matched context lengths would change weights and training dynamics, preventing attribution to visible history alone. That adaptation question is addressed in Table 3. Thus, the train–test context mismatch is an intended intervention rather than an uncontrolled variable. We will clarify that Table 5 measures history utilization, not optimal short-context performance.
As explained above, each world token is independently encoded from the current observation, so the causal Transformer does not map history into world tokens. It only contextualizes the token sequence for action prediction, and the fixed-checkpoint history interventions reported above show that this contextualization does not ignore history.
As described above, WorldToken is not intended to be a dedicated memory mechanism. Its one-token-per-timestep interface supports long explicit contexts and provides memory-like functionality, as validated by the fixed-checkpoint history interventions.