acceptodds
Withdrawn from ICLR 2027

MiX-World: A Unified World Simulator for Manipulators and Locomoting Humanoids

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

16 views

Abstract

Action-conditioned world models forecast how visual observations evolve in response to robot actions, yet existing approaches are largely confined to a single robot embodiment. Although robots differ substantially in morphology and control, they interact with the same physical world, suggesting that experience collected across embodiments may provide complementary supervision. However, bringing these experiences together in a single model is challenging. Low-level actions differ across embodiments in both control semantics and action-space parameterization, while trajectories span different physical timescales. In this paper, we present MiX-World, a unified action-conditioned world simulator for heterogeneous robot embodiments. To our knowledge, it is the first such model to include full-body locomoting humanoids in joint training with mobile-base and fixed-base bimanual manipulators. MiX-World uses Semantically Structured Conditioning (SSC) to represent heterogeneous robot controls, while a shared generator predicts synchronized future observations from multiple views. To capture both transient interactions and long-horizon evolution, MiX-World introduces Event-Anchored Scheduling (EAS) with Time-Anchored RoPE (TA-RoPE). MiX-World achieves better aggregate visual prediction performance than the evaluated baselines at both short and long horizons. With one demonstration per task and 100 adaptation updates, pretraining also improves full-rollout prediction on three unseen embodiments. Controlled temporal ablations further demonstrate that variable-time training improves both short-timescale event prediction and long-horizon forecasting. MiX-World thus provides a scalable foundation for policy evaluation, interactive data collection, and policy post-training across diverse robot embodiments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotics World model, Cross-Embmbodiment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FdTbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MiX-World, a unified action-conditioned world simulator trained jointly on full-body locomoting humanoids, mobile-base bimanual manipulators, and fixed-base bimanual manipulators. It is initialized from the Wan2.1-Fun-V1.1-14B-InP latent video generator and uses Semantically Structured Conditioning (SSC) to map heterogeneous actions into a common vector. Event-Anchored Scheduling (EAS) with Time-Anchored RoPE (TA-RoPE) helps it capture both transient interactions and long-horizon evolution. At both short (4.8 s) and long (96 s) horizons, MiX-World achieves better aggregate visual prediction than embodiment-specific baselines. Cross-embodiment pretraining also improves adaptation to unseen tasks and to three unseen embodiments in RoboTwin 2.0 and LIBERO over the baseline without robot pretraining. Post-training with its generated trajectories and the original data improves the real-robot success rates of three policies by 15.2–21.2 percentage points.

Critical strengths
  1. The paper trains full-body locomoting humanoids together with mobile-base and fixed-base bimanual manipulators in a single model. It generates multiple camera views jointly and predicts 96 s ahead through autoregressive rollout. Few prior action-conditioned world models do all of these at once. The training corpus, roughly 10,000 hours of robot interaction, is also large.

  2. Convincing transfer results with detailed reporting. With the same architecture, joint training is better than single-embodiment training for all three robot families. On dual-arm embodiments unseen in pretraining, MiX-World initialization with only 10 demonstrations per task outperforms the baseline without robot pretraining trained on 100 demonstrations per task. In the adaptation experiments, each comparison between MiX-World and the baseline uses identical target demonstrations and matched optimizer updates. The paper also reports standard deviations over multiple inference seeds, along with per-task and per-view results.

Critical weaknesses
  1. The effect of cross-embodiment pretraining on transfer is not isolated. Table 5 shows that, on the robot families seen in training, joint training predicts better than single-embodiment training. The paper’s stronger claim, however, is that cross-embodiment pretraining helps adaptation to unseen tasks and unseen embodiments (§3.3 and the contributions). This claim is only tested against the baseline without robot pretraining. That shows pretraining on robot data helps, but not how much of the gain comes from mixing data across embodiments. The single-embodiment models in Table 5 are never used as initializations for adaptation. Running the same adaptation from them would provide this control. In addition, Table 5 evaluates seen tasks, and several of them also appear in similar form in other robots’ data, for example cushion tidying and sock folding. These gains may therefore overstate how much joint training helps in new scenes.

  2. The post-training gains lack a control. The policies are post-trained on generated trajectories together with the original data, but there is no control that continues training on the original data alone with the same number of updates. The higher success rates may therefore come from the extra training, not necessarily from the generated data.

  3. The baseline comparison is neither fully fair nor transparent. The authors retrain the three baselines on their own data but do not explain how. The paper does not say how the baselines take the humanoid’s action input (SONIC tokens) or the base and lift actions. DreamDojo does not natively support multi-view generation, and the paper does not explain how its three-view predictions were produced. Table 1 only reports averages over the three embodiments, so readers cannot see which method is better on each one. In Table 5, the single-embodiment version of MiX-World, with the same architecture, reaches an average short-horizon PSNR of 20.63, SSIM of 0.764, and LPIPS of 0.209. That is worse than both DreamDojo (22.76 / 0.807 / 0.192) and Ctrl-World (21.70 / 0.772 / 0.204) on all three metrics. The two tables match training budgets differently, so this is only a rough comparison. Still, it suggests that the short-horizon advantage comes mainly from the extra data in joint training, rather than from EAS, TA-RoPE, or the architecture itself. A fairer comparison would train the baselines on the same mixed corpus.

Questions for authors
  1. Could you add a control pretrained on a single robot family with the same compute, and adapt it to the RoboTwin dual-arm embodiments and LIBERO? This would separate the effect of robot-data pretraining from that of cross-embodiment pretraining. The single-embodiment models in Table 5 could be used directly as initializations.

  2. What are the real-robot success rates if the policies are post-trained on the original data only, with the same number of policy updates? How many trials does each entry in Table 3 use, and what are the confidence intervals?

  3. Were the baselines initialized from their publicly released weights, and what backbone and model size does each use (for example, the 2B or 14B version of DreamDojo)? How do they take SONIC tokens and the base and lift actions as input? DreamDojo does not natively support multi-view generation, so how were its three-view predictions produced? Could you report per-embodiment baseline results with variance?

Reviewer_nvDxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MiX-World, a unified action-conditioned video world model jointly trained on three robot families: locomoting humanoids, mobile-base bimanual manipulators, and fixed-base bimanual manipulators. The method introduces Semantically Structured Conditioning (SSC) to represent heterogeneous action spaces, together with Event-Anchored Scheduling (EAS) and Time-Anchored RoPE (TA-RoPE) to accommodate variable physical timescales and preserve short interaction events during long-horizon prediction. Built on a pretrained Wan video generator, the model jointly predicts synchronized multi-view observations.

The paper evaluates visual prediction quality, transfer to unseen tasks and embodiments, temporal modeling, scene generalization, and downstream policy evaluation and post-training. The reported results suggest that joint cross-embodiment pretraining improves visual prediction, reduces the target demonstration budget required for adaptation, and provides generated trajectories that improve real-robot policy success rates.

The problem is timely and relevant to the development of general-purpose robot world models. The inclusion of full-body humanoid locomotion alongside fixed- and mobile-base manipulation is an interesting direction, and the paper presents a broad set of experiments. However, I have concerns regarding the strength of the methodological novelty, the interpretation of visual prediction metrics as evidence of physical simulation fidelity, and the experimental evidence supporting the downstream utility of the learned world model.

Critical strengths
  1. A relevant and challenging problem formulation.

Unifying humanoid locomotion and manipulation within a shared action-conditioned world model is a meaningful research direction. The different action semantics, camera configurations, and temporal characteristics make this more challenging than simply combining visually similar manipulation datasets. The inclusion of humanoid head and wrist views is particularly relevant because whole-body motion induces substantial changes in camera pose.

  1. A potentially useful cross-embodiment pretraining result.

The paper reports improvements over embodiment-specific world-model baselines at both short and long horizons. It also evaluates transfer to three unseen embodiments, including dual-arm Franka, dual-arm UR5, and single-arm Franka in LIBERO. The reported result that adaptation with 10 demonstrations can outperform a baseline using 100 demonstrations is promising, provided that the comparison is sufficiently controlled and statistically reliable.

  1. Thoughtful treatment of heterogeneous actions and temporal sampling.

SSC explicitly distinguishes valid zero-valued commands from action dimensions that do not exist for a particular embodiment. EAS attempts to preserve short-lived transitions while sampling trajectories across multiple temporal scales, and TA-RoPE encodes elapsed physical time rather than relying exclusively on frame indices. These designs address practical issues that arise when jointly training on robot datasets with different control interfaces and recording rates.

Critical weaknesses
  1. Limited methodological novelty.

MiX-World combines a pretrained Wan video generator with Semantically Structured Conditioning (SSC), Event-Anchored Scheduling (EAS), and Time-Anchored RoPE (TA-RoPE). While these designs address practical challenges in heterogeneous action conditioning and temporal modeling, their conceptual novelty beyond adapting existing video-generation architectures remains somewhat limited. The experiments demonstrate the benefits of individual components, but do not fully isolate the contribution of cross-embodiment learning from increased data diversity and training scale.

  1. Insufficient evidence of physical simulation fidelity.

The evaluation primarily relies on visual metrics such as PSNR, SSIM, and LPIPS. Although the results demonstrate improved visual prediction, these metrics do not establish that the model accurately captures robot-environment interactions or maintains physically consistent states during long-horizon rollouts. The unseen-scene experiments are also primarily qualitative. Additional task-relevant evaluations, such as robot-state prediction, object displacement, or interaction outcomes, would strengthen the claim that MiX-World serves as a reliable world simulator rather than primarily an action-conditioned video generator.

  1. Limited validation of downstream benefits.

The policy evaluation and post-training results are promising, with reported real-robot success-rate improvements across three policies. However, evaluating only three policies provides limited evidence for general policy evaluation, and the paper does not sufficiently isolate the benefits of generated trajectories from additional optimization or training data. Stronger data-matched baselines and more detailed statistical reporting would help establish the reliability and generality of these downstream gains.

Questions for authors
  1. Physical validity: Can the authors provide quantitative evaluations of predicted robot states, object motion, contact events, and task outcomes against real executions, beyond PSNR, SSIM, and LPIPS?

  2. Data and capacity controls: How much of the improvement comes from cross-embodiment supervision itself, as opposed to increased data diversity, total training compute, or the pretrained video backbone? Can the authors provide controlled comparisons that isolate these factors?

  3. Long-horizon reliability: How do prediction errors and task-relevant state inconsistencies accumulate over 96-second autoregressive rollouts? Does action controllability remain reliable late in the rollout?

  4. Policy post-training: How does post-training on MiX-World-generated trajectories compare with training for the same number of updates on real data alone, or with equivalent data generated by another world model? Are the success-rate improvements statistically significant?

  5. Evaluation scope: How many tasks, trials, and independent evaluation seeds support the policy-ranking and post-training claims? How robust are these conclusions beyond the three reported policies?

Reviewer_8uU6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MiX-World, an action-conditioned video world model jointly trained on locomoting humanoids and mobile-base and fixed-base bimanual manipulators. The method combines semantically structured action conditioning with event-anchored sampling and physical-time conditioning to accommodate heterogeneous control interfaces and interaction timescales. Experiments report improved visual prediction at short and long horizons, improved adaptation to unseen tasks and manipulator embodiments, and real-robot policy improvements following post-training with generated trajectories.

Critical strengths

Consistent prediction and adaptation gains across heterogeneous embodiments. The same-architecture comparison in Table 5 shows that joint training improves visual prediction over single-embodiment training across all three robot families. Matched-budget adaptation experiments further demonstrate gains on unseen tasks within pretrained families and on unseen dual-arm Franka, dual-arm UR5, and single-arm Franka embodiments. Together, these results support the practical usefulness of a shared robot prediction prior.

Critical weaknesses
  • The specific contribution of humanoid experience is not isolated. Although Table 5 supports joint training over single-embodiment training, it does not establish whether including humanoid data provides additional benefits beyond mixing manipulator datasets or increasing training-data diversity.
  • Humanoid evaluation provides limited evidence of whole-body control fidelity and transfer. Humanoid prediction is evaluated primarily through visual similarity metrics and qualitative action interventions. These results do not directly establish accurate locomotion progression, whole-body trajectory following, or contact consistency under the conditioning inputs.
Questions for authors
  • Can you provide a controlled comparison of pretraining with and without humanoid data, matching the architecture, optimization budget, and total training-data volume? This would clarify whether humanoid experience benefits prediction and adaptation beyond what is obtained from manipulator-only pretraining.
  • What quantitative evidence supports accurate humanoid locomotion and whole-body action following beyond visual similarity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.