- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
he paper proposes AW-0, a world action model (WAM) that adds a “code expert” to a video-plus-action generative policy. A Mixture-of-Transformers backbone holds three experts: a code expert (a diffusion LLM with a frozen Qwen3-VL ViT), a video expert (Cosmos-Predict2.5-2B), and an action DiT. They are denoised jointly: masked discrete diffusion for code, flow matching for video and actions, with blockwise-causal attention ordered code -> video -> action. The code consists of Python “skill programs” written against primitive robot APIs. An agentic pipeline produces these labels: Gemini 3.1 Pro writes them for demonstration slices, and a Claude Opus 4.8 coordinator distills recurring patterns into a frozen library of 50 skills (23 for YAM, 27 for egocentric human data). The model is pretrained on about 2,900 hours of YAM teleoperation and egocentric human video annotated this way. By default the generated code is not executed and serves only as conditioning. An optional “hybrid” mode runs the code Code-as-Policies style when it passes validity gates, and otherwise falls back to the action chunk. On RoboTwin 2.0 (50 tasks), AW-0 reports 80.8% clean and 20.6% randomized success, versus 64.8% and 27.7% for the strongest respective baselines. On 8 real-world tasks across two embodiments (parallel grippers and 22-DoF dexterous hands), it reports a 71.1% average rubric score versus 46.0% for the best baseline. Ablations cover attention ordering, ViT vs. VAE conditioning, label content (comments, skills, language subgoals) and joint vs. autoregressive decoding, the last with latency measurements.
Critical strengths
-
Putting programmatic skill planning inside a jointly denoised video-action model, rather than wrapping a policy with an external agent, is a fresh idea. Code serves both as conditioning and as an optionally executable artifact.
-
C->V->A beats V->C->A and V->A. Full code beats code without comments, code without skills, and natural-language subgoals on most tasks. This gives fairly direct evidence that structured code, and not just extra language supervision, helps.
-
Joint denoising seems to outperform autoregressive code generation on 7 of 8 tasks while being 2.4× faster, and it seems to add only 0.04 s over the code-free WAM.
-
The tasks span two quite different embodiments, including dexterous 22-DoF hands, and include tasks needing conditional logic and counting (Replace Lightbulb, Flip Scoreboard, Arrange Fruits), where the semantic benefit of code is plausible.
-
Appendix openly reports that code drives only 5–9% of steps overall, and that hybrid mode costs 17.7 points in the clean setting.
-
The annotation pipeline seems to covers about 2.07M subtask slices, with a documented coordinator protocol, skill-admission criteria, and cache-based reuse.
Critical weaknesses
-
By my count, 36 of the 50 references are cited only as arXiv preprints, 12 point to peer-reviewed venues, and 2 are industry model cards. This matters because the arXiv-only set includes the things the paper’s validity rests on: a) every WAM and VLA baseline it reimplements (DreamZero, EgoScale, WLA); b) the evaluation benchmark (RoboTwin 2.0); c) core building blocks (SDAR, Cosmos-Predict2.5, Qwen3-VL, Wan2.2, MoT); d) the explanations for two key results. Both the drop under randomization and the ViT-over-VAE result are justified solely by Yan et al. (arXiv 2608.05903), a preprint posted roughly one month before submission. Several cited preprints (2606., 2607., 2608.*) are essentially concurrent and unvetted. Separately, several works that are peer-reviewed are cited as arXiv only. Positioning against unreviewed work weakens the novelty and SOTA claims, and the bibliography needs a careful pass.
-
AW-0 is pretrained on about 1,900 hours of YAM data, the same embodiment as half the real-world tasks, while baselines start from their public checkpoints. Training budgets also differ widely: DreamZero gets 1 epoch per task, while π0.5, EgoScale and WLA get 50 epochs. The 25-point real-world gap therefore cannot be attributed to the method. This suggests that there is a confounded comparison with baselines.
-
The hybrid-mode comparison seems not controlled. Indeed, in RoboTwin, hybrid mode is trained on demonstrations collected by executing CaP programs, which differ from the standard demos used for the default mode and the baselines. The real-world hybrid evaluation covers only one task.
-
The headline simulation claim depends on averaging. Under randomization, AW-0 is below π0.5. “Highest average” seems to come from combining clean and random scores. The robustness claim in the abstract rests on hybrid mode, whose training data differs (see above).
-
The code-free variant results to be below the DreamZero baseline, yet adding code gives a gain. This raises the question of whether the code labels carry privileged structure: sensor-derived gripper-event sequences, explicit “STAGE k/N” headers, and loop counts read from frames.
-
The quality of the produced code seems quite low, and the quality of the annotations seems unmeasured. Indeed, the pretrained code expert has an entity F1 of 0.534 and a primitive F1 of 0.365. No human audit of the Gemini/Claude labels is reported, even though labels contain “[auto-repair]” insertions. The method’s success may come from code acting as a structured conditioning signal rather than as a correct plan, which undercuts the “agentic reuse” framing.
-
The paper lacks of a proper statistical rigor. Indeed, the paper does not seem to report any seeds, standard errors or significance tests. Real-world metrics are additive partial-credit rubrics, not success rates. Some task names overstate difficulty: “Solve Rubik’s Cube” is a single quarter-turn un-scramble.
-
The pipeline depends on proprietary frontier models (Gemini 3.1 Pro, Claude Opus 4.8), a private 2,900-hour corpus, and 48-GPU pretraining, and there is no release commitment for data, labels or code. THus, the reproducibility is questionable.
-
The skill-reuse claim is under-tested. The YAM and human libraries are disjoint. Cross-embodiment or cross-task reuse of skills, and agentic re-execution beyond one real task, are not demonstrated.
Questions for authors
Q0) Please address all the criticisms above if not properly transformed in questions below.
Q1) Could you replace preprint citations with peer-reviewed versions where they exist? For the claims that currently rest only on recent arXiv work (pixel-level OOD degradation, VAE features being poorly suited to semantic tasks), could you provide your own supporting experiments?
Q2) Were the V->A, V->C->A, subgoal and AR ablations all pretrained on the same 2,900-hour corpus with identical compute? If not, what does each variant see during pretraining?
Q3) Can you report baselines fine-tuned (or ideally co-pretrained) on the same YAM pretraining data, and with matched epochs and compute, so that the method contribution can be separated from the data contribution?
Q4) How much of the code gain survives if you strip privileged information from the labels (gripper-event sequences, stage indices, frame-derived loop counts), keeping only skill calls?
Q5) For RoboTwin hybrid mode, what is AW-0’s default-mode performance when trained on the same CaP-collected demonstrations? Conversely, how do baselines perform when trained on them?
Q6) What fraction of automatically generated labels is correct according to a human audit on a random sample? How often does the “[auto-repair]” path trigger?
Q7) Please report standard errors over rollouts or seeds, binary task-success rates alongside rubric scores for the real-world tasks, and the scrambled-cube distribution used for “Solve Rubik’s Cube”.
Q8) The memory cache (K slots, replacement probability p) and the noise-resampling mixture (α = 0.5) are not ablated. What are their individual contributions?
Q9) How many denoising steps and code-canvas tokens underlie the 0.23 s latency, and how does performance trade off against fewer steps?
Q10) Is any skill invoked across embodiments or on unseen downstream tasks? Can the library be extended at fine-tuning time without retraining the code expert?
Q11) Which diffusion LLM initializes the code expert (LLaDA or SDAR)? Which parameters are frozen versus trained during downstream fine-tuning?
Q12) Will the annotated program dataset, skill library, API definitions and checkpoints be released?
Thank you for the clarifications and additional experiments, which substantially address my main concerns. I will maintain my positive recommendation.
Reviewer KLJm also raised important concerns about the different training datasets used in the original execution-mode comparison and the use of rubric scores in the real-world evaluation. The authors’ controlled comparisons and additional binary success rates with uncertainty estimates provide essential context for interpreting the results. I strongly encourage incorporating both responses into the revised manuscript, together with the updated robustness discussion and clarified task definitions.