- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces GlanceWAM, a world-action model that separates a slow asynchronous visual-lookahead generator from a fast action-chunk policy. The method uses one shared latent video DiT, a three-class attention mask intended to prevent future-lookahead leakage, and random lookahead horizons with explicit offset conditioning to handle aging predictions. Experiments report strong results on RoboCasa, LIBERO, and five real-robot tasks, alongside low hold-phase action latency.
Critical strengths
- The paper is well motivated, communicates the distinction between co-training with a clean, teacher-forced lookahead token and deployment with an asynchronously generated latent lookahead.
- The paper makes a useful systems-level point that a world-model-derived visual target need not be recomputed on every control cycle.
Critical weaknesses
- The main simulation experiments in Section 4.1 pause the simulator during lookahead generation, effectively assuming zero refresh delay. Although the authors demonstrate asynchronous real-robot deployment and report a pipelined experiment in Section 4.5, these results do not fully characterize the effects of delayed, missed, or stale lookahead updates. Since the action head explicitly conditions on the remaining temporal offset, delayed refreshes may introduce inconsistencies between the lookahead representation and its associated offset.
- During training, the action head is conditioned on clean lookahead frames extracted from demonstration trajectories. At inference, however, it receives lookahead latents generated by the video model, which may contain inaccurate object positions, incorrect spatial relationships, or other prediction errors. This issue is particularly important because Table 3a demonstrates that the policy relies heavily on lookahead conditioning. The authors should investigate the relationship between lookahead prediction quality and downstream task success, and evaluate whether training with generated or deliberately perturbed lookaheads improves robustness.
Questions for authors
- In the actual asynchronous implementation, what happens when the new lookahead is not ready at the scheduled refresh time? Does the system hold the previous lookahead, fall back to no-lookahead mode, or block?
- How does performance change if the action head is trained with generated lookaheads, or a mixture of clean and generated lookaheads, rather than only teacher-forced demonstration frames?
- Does the bidirectional-mask model permit clean lookahead tokens to influence future-target prediction during training? Can the authors include a leakage-free alternative mask ablation that isolates the effect of representation isolation from a severe train-test availability mismatch?
4 comments
We value your feedback and are eager to address any further questions or concerns you may have. If you have had a chance to review our response and have additional thoughts, we would greatly appreciate your input.
Response for rebuttal
Thank you for the detailed response and the additional experiments. These additions substantially alleviate my main concerns.
The updated asynchronous evaluation and new ablations have alleviated my main concerns. Robustness to missed updates and inaccurate lookaheads remains an open question, which should be acknowledged as a limitation. I have no further questions.
Thank you for carefully reading our response and the new experiments. We are glad that they address your main concerns.
Since these concerns are now resolved, we would be very grateful if you would consider updating your score to reflect your current assessment. It would mean a great deal to us for this submission.
Following your suggestion, we have added both open questions, robustness to missed updates and to inaccurate lookaheads, to Sec. 5 (Limitations and Future Work) of the uploaded revision. Thank you again for the feedback, which has made the paper better.
Reply to Reviewer VUjw
We thank the reviewer for the thoughtful comments. We address each point below.
1) Delayed, missed or stale lookahead updates (W1, Q1)
Delayed and stale lookaheads. Please see item 1 of the global response. We now emulate asynchronous execution in simulation: each lookahead is generated from the observation 0.5 s before the refresh at which it is first used, and the action head is conditioned on the correspondingly shorter offset, so the lookahead and its offset stay consistent. Under this protocol success is unchanged (RoboCasa 71.6% vs. 71.7% with the simulator paused; LIBERO 98.7% vs. 98.8%), and Tables 1–2 now report these numbers. The lead-time study (Sec. 4.5, Tables 4–5) also covers longer delays of 0.75 s and 1 s.
Lookahead not ready at the refresh. In both the real-robot and the simulation setups, the machine runs only this one model, so the generation time varies very little. Generation takes 339 ms at 10 Euler steps on an RTX 4090 (Table 3b), and the 0.5 s lead time leaves enough buffer on top of it. A lookahead that misses its refresh therefore does not arise in our deployment, and we did not need a fallback for it.
2) Clean training lookaheads vs. generated lookaheads at test time (W2, Q2)
Lookahead quality vs. success. Sec. 4.5 varies lookahead quality through the number of sampler steps (Table 3b). From 1 to 30 Euler steps, image sharpness changes visibly (Figure 12), yet success stays within 1.9% (71.2% at 1 step, 71.7% at 10). The policy reads the low-frequency spatial layout of the lookahead, which the generated latent already gets right, so the gap between clean and generated lookaheads costs little.
Training with generated lookaheads. We ran this experiment (Table 3a, "Training-recipe variants" in Sec. 4.4). Mixing self-generated lookaheads into training is challenging: running the full 10-step denoising for each training sample would slow training down substantially. In the spirit of the chain-of-forward training of Epona [1], we instead generate the lookahead online with a single Euler step from the current model, and use it in place of the clean lookahead for 50% of the samples, after a 1,000-step warm-up with clean lookaheads only. This adds little training cost, and the sampler sweep above shows that a 1-step lookahead is already sufficient at test time. The retrained model reaches 68.7%, against 71.7% with clean lookaheads only. This is one recipe, and the rebuttal period did not allow us to explore others; we conclude that the idea is not ruled out, but it does not work out of the box.
[1] Zhang, Kaiwen, et al. "Epona: Autoregressive Diffusion World Model for Autonomous Driving." ICCV 2025.
3) Leakage-free alternative mask ablation (Q3)
Yes. In the bidirectional model the future-prediction queries can attend to the clean lookahead tokens during training, so that ablation mixes two effects: leakage into the video objective, and loss of the isolation between observation and lookahead.
We added the leakage-free variant you suggested (Table 3a, "Training-recipe variants" in Sec. 4.4). Future queries are still blocked from the lookahead, as in the full mask; only the observation queries are allowed to attend to it. This retrain reaches 55.6% on RoboCasa kitchen, against 71.7% for the model with the full mask and 47.0% for the bidirectional model. Removing the observation–lookahead isolation alone, with no leakage into future prediction, therefore already costs 16.1%, and the bidirectional model loses a further 8.6%.