- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MiX-World, a unified action-conditioned world simulator trained jointly on full-body locomoting humanoids, mobile-base bimanual manipulators, and fixed-base bimanual manipulators. It is initialized from the Wan2.1-Fun-V1.1-14B-InP latent video generator and uses Semantically Structured Conditioning (SSC) to map heterogeneous actions into a common vector. Event-Anchored Scheduling (EAS) with Time-Anchored RoPE (TA-RoPE) helps it capture both transient interactions and long-horizon evolution. At both short (4.8 s) and long (96 s) horizons, MiX-World achieves better aggregate visual prediction than embodiment-specific baselines. Cross-embodiment pretraining also improves adaptation to unseen tasks and to three unseen embodiments in RoboTwin 2.0 and LIBERO over the baseline without robot pretraining. Post-training with its generated trajectories and the original data improves the real-robot success rates of three policies by 15.2–21.2 percentage points.
Critical strengths
-
The paper trains full-body locomoting humanoids together with mobile-base and fixed-base bimanual manipulators in a single model. It generates multiple camera views jointly and predicts 96 s ahead through autoregressive rollout. Few prior action-conditioned world models do all of these at once. The training corpus, roughly 10,000 hours of robot interaction, is also large.
-
Convincing transfer results with detailed reporting. With the same architecture, joint training is better than single-embodiment training for all three robot families. On dual-arm embodiments unseen in pretraining, MiX-World initialization with only 10 demonstrations per task outperforms the baseline without robot pretraining trained on 100 demonstrations per task. In the adaptation experiments, each comparison between MiX-World and the baseline uses identical target demonstrations and matched optimizer updates. The paper also reports standard deviations over multiple inference seeds, along with per-task and per-view results.
Critical weaknesses
-
The effect of cross-embodiment pretraining on transfer is not isolated. Table 5 shows that, on the robot families seen in training, joint training predicts better than single-embodiment training. The paper’s stronger claim, however, is that cross-embodiment pretraining helps adaptation to unseen tasks and unseen embodiments (§3.3 and the contributions). This claim is only tested against the baseline without robot pretraining. That shows pretraining on robot data helps, but not how much of the gain comes from mixing data across embodiments. The single-embodiment models in Table 5 are never used as initializations for adaptation. Running the same adaptation from them would provide this control. In addition, Table 5 evaluates seen tasks, and several of them also appear in similar form in other robots’ data, for example cushion tidying and sock folding. These gains may therefore overstate how much joint training helps in new scenes.
-
The post-training gains lack a control. The policies are post-trained on generated trajectories together with the original data, but there is no control that continues training on the original data alone with the same number of updates. The higher success rates may therefore come from the extra training, not necessarily from the generated data.
-
The baseline comparison is neither fully fair nor transparent. The authors retrain the three baselines on their own data but do not explain how. The paper does not say how the baselines take the humanoid’s action input (SONIC tokens) or the base and lift actions. DreamDojo does not natively support multi-view generation, and the paper does not explain how its three-view predictions were produced. Table 1 only reports averages over the three embodiments, so readers cannot see which method is better on each one. In Table 5, the single-embodiment version of MiX-World, with the same architecture, reaches an average short-horizon PSNR of 20.63, SSIM of 0.764, and LPIPS of 0.209. That is worse than both DreamDojo (22.76 / 0.807 / 0.192) and Ctrl-World (21.70 / 0.772 / 0.204) on all three metrics. The two tables match training budgets differently, so this is only a rough comparison. Still, it suggests that the short-horizon advantage comes mainly from the extra data in joint training, rather than from EAS, TA-RoPE, or the architecture itself. A fairer comparison would train the baselines on the same mixed corpus.
Questions for authors
-
Could you add a control pretrained on a single robot family with the same compute, and adapt it to the RoboTwin dual-arm embodiments and LIBERO? This would separate the effect of robot-data pretraining from that of cross-embodiment pretraining. The single-embodiment models in Table 5 could be used directly as initializations.
-
What are the real-robot success rates if the policies are post-trained on the original data only, with the same number of policy updates? How many trials does each entry in Table 3 use, and what are the confidence intervals?
-
Were the baselines initialized from their publicly released weights, and what backbone and model size does each use (for example, the 2B or 14B version of DreamDojo)? How do they take SONIC tokens and the base and lift actions as input? DreamDojo does not natively support multi-view generation, so how were its three-view predictions produced? Could you report per-embodiment baseline results with variance?