acceptodds
Under review as a conference paper at ICLR 2027

Real Where It Matters: Sim–And-Real Co-Training for Action-Conditioned World Models

Abstract

Action-conditioned world models simulate future dynamics conditioned on robot actions, but their generalization is constrained by the limited coverage of real-world robot datasets. Existing datasets are dominated by successful human-teleoperated trajectories and shoulder-mounted exocentric viewpoints, leaving off-nominal interactions and broader camera viewpoints sparsely represented. Consequently, models trained on these data struggle to predict outcomes of non-optimal actions and generalize across camera viewpoints in the real world. In contrast, physics simulators can generate off-nominal interactions, such as retries and inter-object contacts, across diverse camera viewpoints at scale. In this paper, we investigate whether large-scale simulation data can complement finite real-world robot data for training action-conditioned world models for robotic manipulation. We jointly train a world model, dubbed MixWorld, on real (  95k episodes) and simulated (  550k episodes) trajectories, where real data ground predictions in the deployment domain while simulation provides substantially broader coverage of transition dynamics and camera viewpoints. Our results show that sim-and-real co-training substantially improves video prediction quality across computational metrics, model-based evaluations, and the comprehensive WorldArena benchmark. More importantly, these improvements translate into consistent gains in downstream applications, including policy evaluation and inference-time search. For policy evaluation, our world model improves Pearson correlation () by 109.3% and Spearman correlation () by 142.4%, while reducing Mean Maximum Rank Violation (MMRV) by 18.2% relative to the base world model. For Best-of-N inference-time search, our world model increases average success rates by 10.0%–33.3% over the base policy and 16.7%–30.0% over search with the base world model across the two tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, sim-and-real co-training, robot manipulation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_Gsa8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether jointly training on real and simulated data improves action-conditioned robot world models. MixWorld starts from a Ctrl-World checkpoint pretrained on DROID and incorporates approximately 95,000 real and 550,000 simulated trajectories to broaden coverage of off-nominal interactions and camera viewpoints. The authors compare full-model mixture training with LoRA adaptation, with the latter additionally using a latent dynamics consistency loss and asymmetric action dropout. Evaluation includes video prediction on multiple real-world datasets, WorldArena, human preferences, real-world policy evaluation, and Best-of-N action selection. The reported results show improvements in several prediction metrics and downstream tasks.

Critical strengths

A practically relevant problem. Training predominantly on successful demonstrations can leave world models unable to faithfully predict the consequences of suboptimal actions. The focus on failures, retries, and inter-object interactions is directly relevant to using world models for policy evaluation and planning.

Evaluation extends beyond visual prediction quality. In addition to video metrics and human preferences, the paper examines agreement between simulated and real policy performance and tests world-model-assisted action selection on a physical robot. These experiments provide preliminary evidence of practical benefits from co-training.

Critical weaknesses

Insufficient novelty relative to existing work. Real–simulation co-training is well established in robot policy learning. More directly, SimDist already uses off-nominal simulated interactions to train transferable robot world models, while ReSim combines real data with simulated non-expert behavior to improve action-conditioned prediction. Although this paper extends the setting to larger-scale, multi-scene, multi-view robot video modeling, the results do not sufficiently establish a new methodological contribution or generalizable insight beyond these existing principles. Combining straightforward data mixing, LoRA, and an additional consistency objective does not by itself clearly establish novelty.

The experiments insufficiently isolate the effects of the added data and training choices. There is no matched-budget real-data-only continuation baseline, nor adequate separation of simulation-data quantity, off-nominal interaction coverage, and viewpoint diversity. The LoRA variant also changes the mixing ratio and learning rate while introducing consistency loss and action dropout. Consequently, it remains unclear whether the gains arise from a specific co-training mechanism or simply from additional data, computation, or other recipe changes. This also weakens the paper’s novelty as an empirical study.

Questions for authors

What is the central new finding beyond SimDist, ReSim, and prior robot co-training studies? Beyond scale and application setting, which results establish a previously unknown method or insight?

4 comments
AuthorsOpenReview ↗

Reply to Reviewer Gsa8 (1)

We thank the reviewer for the thoughtful comments regarding novelty and experimental attribution.

W1: Distinction from previous work

We agree that sim-and-real co-training is not a new principle, and that SimDist and ReSim provide important precedents for leveraging simulated interactions to improve world models. Although our work shares the motivation of leveraging simulation to expand interaction coverage, we would like to highlight that it investigates a substantially different learning setting and provides complementary empirical findings for the robot manipulation world-model domain.

We also agree that data mixing, LoRA, and a consistency objective are not in themselves a methodological contribution. Our contribution is the empirical findings listed under Q1, and we will revise the paper to state this explicitly.

SimDist studies simulation pretraining followed by real-world adaptation in task-specific manipulation settings. We study joint training on large-scale heterogeneous data: 95k real DROID trajectories and 550k simulated trajectories spanning over 60k environments and 7k assets. This poses a different generalization question: whether interaction experience from diverse simulated environments transfers to real scenes and tasks that have no simulated counterpart.

ReSim establishes the co-training principle for autonomous driving video models, with a single ego-camera and ego-trajectory actions. In manipulation, what must transfer is different: contact outcomes such as missed grasps and inter-object interactions, observed from multiple third-person and wrist viewpoints.

Policy learning with sim-and-real co-training shares its prediction target across domains: both simulated and real inputs map to actions, so the visual gap is confined to the input. A video world model must generate the observations themselves, so the sim-to-real gap lies in the prediction target. Results from policy co-training therefore do not imply that co-training helps a world model; simulated data could equally degrade real-domain generation.

Q1: Central new findings that are not revealed in precedents such as SimDist and ReSim.

  1. Simulated off-nominal interactions and viewpoints transfer to real-world prediction. Against a matched real-only control added in rebuttal, co-training is better on PSNR, SSIM, and LPIPS in both views and on exocentric FVD on our real evaluation set of missed grasps, inter-object interactions, and randomized exocentric cameras.

  2. A simple mixture is sufficient. Mixing simulated and real data, with no architectural change or domain-specific module, already improves prediction on state transitions that are underrepresented in real-robot data.

  3. More real data of the same kind is not a substitute. Real-only continuation peaks at 10k steps and then falls below the pretrained checkpoint on the off-nominal evaluation set, while co-training stays stable (W2, Part 1).

  4. The world-model gains carry over to downstream manipulation-related tasks. Relative to the pretrained Ctrl-World, Best-of-N search with our model raises success over the base policy by 19.2 points across four tasks (63/120 vs. 40/120), and by 16.7 points over search with Ctrl-World. Policy-evaluation correlations also improve (Pearson r from 0.196 to 0.390), though the confidence intervals of the differences include zero.

We will update the manuscript to differentiate our work from related work more clearly.

AuthorsOpenReview ↗

Reply to Reviewer Gsa8 (2)

W2: Isolation of effects

Part 1: Real-only continued-training baseline.

We continued training the same Ctrl-World checkpoint on DROID only, with the same learning rate, batch size, and other configurations as Naive Mix (i.e., α=0\alpha=0 vs. 0.9). Due to computational and time limit, we compare their checkpoints every 5k steps up to 35k (matched compute for 35k training steps). Checkpoints for all models are selected by one criterion: the most wins over the pretrained Ctrl-World across the 30 metric cells (3 eval sets ×\times 2 views ×\times 5 metrics), since our expectation is that finetuning can bring generalization to counterfactual cases only seen in simulation while maintaining the performance on near-distribution eval data. This selects DROID-only at 10k (29/30) and Naive Mix (sim:real=90:10) at 35k (26/30).

Eval Set 3 (missed grasps, randomized exocentric cameras, inter-object interactions), selected checkpoints:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoCtrl-World19.770.7340.12438.49600.9
ExoDROID-only (10k)20.25‾\underline{20.25}0.740‾\underline{0.740}0.117‾\underline{0.117}32.35427.9‾\underline{427.9}
ExoNaïve Mix (35k)20.850.7550.10434.15‾\underline{34.15}337.2
WristCtrl-World19.030.6420.30922.09220.8
WristDROID-only (10k)19.33‾\underline{19.33}0.644‾\underline{0.644}0.287‾\underline{0.287}18.84212.0
WristNaïve Mix (35k)19.870.6640.27619.43‾\underline{19.43}213.1‾\underline{213.1}

Eval Set 3, Exo, PSNR$\uparrow$ / FVD$\downarrow$ by training step to show training dynamics:

Model5k10k15k20k25k30k35k
DROID-only20.03 / 486.020.25 / 427.920.02 / 499.120.04 / 501.519.93 / 488.119.90 / 489.819.59 / 655.3
Naïve Mix20.77 / 339.720.90 / 348.720.89 / 353.920.87 / 341.521.12 / 343.121.10 / 336.920.85 / 337.2

First, we can see that sim-and-real co-training improves prediction on Eval Set 3. At the selected checkpoints, Naive Mix is better than DROID-only on PSNR, SSIM, and LPIPS in both views and on Exo FVD (337.2 vs. 427.9). DROID-only has the lower FID in both views, and Wrist FVD is on par (212.0 vs. 213.1).

Second, we observe that, as training steps increase, DROID-only training degrades on Eval Set 3 while co-training does not. In the Exo view, DROID-only peaks at 10k on all five metrics and falls below the pretrained checkpoint by 35k (19.59 PSNR, 655.3 FVD). Naive Mix stays stable through 35k.

Third, it merits attention that co-training actually uses less real-robot data. At α\alpha=0.9, Naive Mix at 35k has seen about 3.5k steps' worth of DROID samples, roughly a third of what DROID-only has seen at 10k.

Fourth, real-only training helps on near-distribution Eval Sets 1 and 2 more than co-training does. On these sets, which are closer to the DROID distribution, DROID-only has lower FID and FVD than Naïve Mix at every step and does not degrade between 5k-35k training steps. Part of the improvement over Ctrl-World reported on these sets is therefore attributable to continued training. We will add DROID-only as a baseline and report that results in Sec.5.2 with updated arguments.

The conclusion from continued training on DROID only vs naive sim-and-real co-training is that: 1. On evaluation sets closer to the training distribution, continued real-only training is as good or better on distribution-level metrics; 2. co-training helps on the conditions it was introduced to cover, off-nominal interactions and novel viewpoints, where continued real-only training degrades.

AuthorsOpenReview ↗

Reply to Reviewer Gsa8 (3)

Part 2: LoRA w/ dynamics consistency loss (DCL) and asymmetric action dropout vs. plain LoRA w/o additional mechanism.

We trained a plain LoRA removing both DCL and asymmetric action dropout, while keeping other configurations the same as LoRA w/ DCL. Under the same checkpoint-selection rule (wins over Ctrl-World across 30 metric cells), plain LoRA is selected at 35k with 26/30 wins; LoRA w/ DCL at 35k reaches 30/30.

Eval Set 3, both at 35k:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoPlain LoRA20.420.7460.11334.23430.0
ExoLoRA w/ DCL20.640.7530.10534.51366.4
WristPlain LoRA19.590.6520.28821.37224.3
WristLoRA w/ DCL19.720.6620.28120.17215.5

On Eval Set 3, adding DCL and dropout improves every metric except Exo FID, with the largest change in Exo FVD (430.0 to 366.4). Plain LoRA's Wrist FVD (224.3) is worse than the pretrained checkpoint (220.8), and with DCL and dropout it is better (215.5).

On Eval Sets 1 and 2, plain LoRA is slightly better than LoRA w/ DCL, since with simulation data, the benefit is concentrated on the off-nominal and novel-viewpoint set.

Finally, we acknowledge the limits of these controls. This ablation establishes the combined effect of DCL and asymmetric action dropout at a fixed mixing ratio and adaptation method, but does not separate the two. Due to compute and time constraints, we also could not separate simulation-data quantity, off-nominal interaction coverage, and viewpoint diversity, sweep the mixing ratio, or match the ratio and learning rate between the full-model and LoRA variants. Since the central question of the paper is whether sim-and-real co-training helps, we will state the lack of these studies as limitations and leave them to future work.

Thank you for the detailed responses and additional experiments. The real-only continued-training baseline and the plain-LoRA comparison help clarify both the contribution of simulation data and the effects of the proposed training mechanisms.

  1. Novelty and scope. The distinction from SimDist and ReSim is clearer. I appreciate the more precise framing of the contribution as empirical evidence for simulation-to-real transfer in world-model prediction. The revised manuscript should consistently reflect this scope.
  2. Attribution to simulation data. The real-only baseline provides useful evidence that the benefits of co-training are concentrated on off-nominal interactions and novel viewpoints. The finding that real-only training remains competitive or better on evaluation sets closer to the training distribution is equally important and should accompany the main claims. The mixed results across metrics also warrant a qualified interpretation.
  3. Attribution to individual mechanisms. The plain-LoRA comparison supports the combined benefit of dynamics consistency loss and asymmetric action dropout in the evaluated setting. As the authors acknowledge, it does not establish their individual contributions. This distinction should remain explicit in the revised manuscript. I will maintain my original score. Nevertheless, I appreciate the effort invested during the rebuttal process and believe that incorporating these revisions will make the work more convincing. I am not requesting further experiments at this stage and leave the final decision to the program committee.
Reviewer_oJJVOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether large-scale simulation data can improve the real-world prediction capabilities of action-conditioned robot world models. The key motivation is that existing real-robot datasets are dominated by successful demonstrations and provide insufficient coverage of off-nominal interactions, failure cases, and diverse camera viewpoints.

The authors propose MixWorld, building upon Ctrl-World and jointly training on approximately 95K real-robot trajectories from DROID and 550K simulated trajectories from MolmoBot-Data. Two co-training strategies are investigated: direct data mixing and LoRA-based adaptation with additional dynamics supervision.

Experiments demonstrate improvements in video prediction quality across multiple evaluation datasets and WorldArena metrics. More importantly, the authors evaluate downstream applications on real robots, showing improved agreement between world-model-based and real-world policy evaluations, as well as improved manipulation success through inference-time Best-of-N action selection.

Critical strengths
  1. An important and practically relevant investigation of world-model training data. The paper addresses a fundamental limitation of current robot world models: insufficient coverage of action-conditioned dynamics, particularly failures and off-nominal interactions. Using simulation to expand the diversity of transitions while retaining real data for domain grounding is a promising direction. The scale of the combined dataset and the demonstrated improvements suggest that simulation can potentially play an important role in training general-purpose robot world models.

  2. Strong emphasis on downstream utility and real-world validation. A major strength is that the evaluation goes beyond conventional video-generation metrics. The authors investigate whether the improved world model can better predict the relative performance of different robot policies and support inference-time action search. The improvement in policy-evaluation correlation and the real-robot Best-of-N results are encouraging. These experiments directly address whether world-model improvements translate into better robot decision-making, which is particularly valuable for this research direction.

Critical weaknesses
  1. The contribution of simulation data is not fully isolated from additional training and other methodological changes. The comparison primarily uses the original pretrained Ctrl-World checkpoint as the baseline, while MixWorld undergoes additional co-training. A compute-matched continuation of Ctrl-World using only real-world data would be important to establish how much improvement specifically comes from simulation rather than additional optimization. Furthermore, the LoRA configuration also introduces dynamics consistency loss and asymmetric action dropout, while the two co-training strategies use different mixing ratios. Without more controlled ablations, it is difficult to attribute the improvements to individual components or understand the optimal training recipe.

  2. The real-world downstream evaluation is promising but relatively limited. Policy evaluation considers a small number of policies and tasks, while inference-time search is evaluated on only two manipulation tasks with a limited number of trials. The improvements are encouraging, but uncertainty estimates would help establish their statistical reliability. Additionally, the simulation data primarily covers rigid-object pick-and-place interactions on a single robot embodiment. It remains unclear how broadly the findings generalize to more complex contact dynamics, deformable objects, dexterous manipulation, and different robot embodiments.

Questions for authors
  1. Can the authors provide a compute-matched real-only continued-training baseline, initialized from the same Ctrl-World checkpoint and trained for the same number of additional steps? This is important to isolate the benefit of introducing simulation data from that of additional training.

  2. How sensitive is performance to the real/simulation mixing ratio? Can the authors separately quantify the effects of simulation data, dynamics consistency loss, and asymmetric action dropout? It would also be interesting to distinguish improvements attributable to broader camera distributions from those attributable to off-nominal interaction dynamics.

  3. Can the authors provide confidence intervals or additional trials for the real-robot policy evaluation and Best-of-N experiments? In particular, how consistently does the improved world model predict failures and rank candidate actions across different tasks and initial configurations?

3 comments
AuthorsOpenReview ↗

Replay to Reviewer oJJV (1)

We thank the reviewer for the constructive feedback. Please see our responses below.

W1/Q1 Real-only continued-training baseline.

We continued training the same Ctrl-World checkpoint on DROID only, with the same learning rate, batch size, and other configurations as Naive Mix (i.e., α=0\alpha=0 vs. 0.9). Due to computational and time limit, we compare their checkpoints every 5k steps up to 35k (matched compute for 35k training steps). Checkpoints for all models are selected by one criterion: the most wins over the pretrained Ctrl-World across the 30 metric cells (3 eval sets ×\times 2 views ×\times 5 metrics), since our expectation is that finetuning can bring generalization to counterfactual cases only seen in simulation while maintaining the performance on near-distribution eval data. This selects DROID-only at 10k (29/30) and Naive Mix (sim:real=90:10) at 35k (26/30).

Eval Set 3 (missed grasps, randomized exocentric cameras, inter-object interactions), selected checkpoints:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoCtrl-World19.770.7340.12438.49600.9
ExoDROID-only (10k)20.25‾\underline{20.25}0.740‾\underline{0.740}0.117‾\underline{0.117}32.35427.9‾\underline{427.9}
ExoNaïve Mix (35k)20.850.7550.10434.15‾\underline{34.15}337.2
WristCtrl-World19.030.6420.30922.09220.8
WristDROID-only (10k)19.33‾\underline{19.33}0.644‾\underline{0.644}0.287‾\underline{0.287}18.84212.0
WristNaïve Mix (35k)19.870.6640.27619.43‾\underline{19.43}213.1‾\underline{213.1}

Eval Set 3, Exo, PSNR$\uparrow$ / FVD$\downarrow$ by training step to show training dynamics:

Model5k10k15k20k25k30k35k
DROID-only20.03 / 486.020.25 / 427.920.02 / 499.120.04 / 501.519.93 / 488.119.90 / 489.819.59 / 655.3
Naïve Mix20.77 / 339.720.90 / 348.720.89 / 353.920.87 / 341.521.12 / 343.121.10 / 336.920.85 / 337.2

First, we can see that sim-and-real co-training improves prediction on Eval Set 3. At the selected checkpoints, Naive Mix is better than DROID-only on PSNR, SSIM, and LPIPS in both views and on Exo FVD (337.2 vs. 427.9). DROID-only has the lower FID in both views, and Wrist FVD is on par (212.0 vs. 213.1).

Second, we observe that, as training steps increase, DROID-only training degrades on Eval Set 3 while co-training does not. In the Exo view, DROID-only peaks at 10k on all five metrics and falls below the pretrained checkpoint by 35k (19.59 PSNR, 655.3 FVD). Naive Mix stays stable through 35k.

Third, it merits attention that co-training actually uses less real-robot data. At α\alpha=0.9, Naive Mix at 35k has seen about 3.5k steps' worth of DROID samples, roughly a third of what DROID-only has seen at 10k.

Fourth, real-only training helps on near-distribution Eval Sets 1 and 2 more than co-training does. On these sets, which are closer to the DROID distribution, DROID-only has lower FID and FVD than Naïve Mix at every step and does not degrade between 5k-35k training steps. Part of the improvement over Ctrl-World reported on these sets is therefore attributable to continued training. We will add DROID-only as a baseline and report that results in Sec.5.2 with updated arguments.

The conclusion from continued training on DROID only vs naive sim-and-real co-training is that: 1. On evaluation sets closer to the training distribution, continued real-only training is as good or better on distribution-level metrics; 2. co-training helps on the conditions it was introduced to cover, off-nominal interactions and novel viewpoints, where continued real-only training degrades.

AuthorsOpenReview ↗

Replay to Reviewer oJJV (2)

W1/Q2: LoRA w/ dynamics consistency loss (DCL) and asymmetric action dropout vs. plain LoRA w/o additional mechanism; Ratio sweep.

We trained a plain LoRA removing both DCL and asymmetric action dropout, while keeping other configurations the same as LoRA w/ DCL. Under the same checkpoint-selection rule (wins over Ctrl-World across 30 metric cells), plain LoRA is selected at 35k with 26/30 wins; LoRA w/ DCL at 35k reaches 30/30.

Eval Set 3, both at 35k:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoPlain LoRA20.420.7460.11334.23430.0
ExoLoRA w/ DCL20.640.7530.10534.51366.4
WristPlain LoRA19.590.6520.28821.37224.3
WristLoRA w/ DCL19.720.6620.28120.17215.5

On Eval Set 3, adding DCL and dropout improves every metric except Exo FID, with the largest change in Exo FVD (430.0 to 366.4). Plain LoRA's Wrist FVD (224.3) is worse than the pretrained checkpoint (220.8), and with DCL and dropout it is better (215.5).

On Eval Sets 1 and 2, plain LoRA is slightly better than LoRA w/ DCL, since with simulation data, the benefit is concentrated on the off-nominal and novel-viewpoint set.

We acknowledge that this ablation establishes the combined effect of DCL and asymmetric action dropout at a fixed mixing ratio and adaptation method but does not separate DCL from asymmetric dropout due to computational and time constraints.

Since the main objective of MixWorld is to show that sim-and-real co-training improves models' performance on counterfactual cases unseen in real robot data but covered in simulation data, we leave the study of ratio sweeps and the best co-training strategy to future work. For the study of data decomposition, our evaluation set covers both camera variation and off-nominal dynamics, but our training runs do not separate them. We leave this decomposition to future work. Finally, more complex contact dynamics, deformable objects, dexterous manipulation, and more robot embodiments are also interesting future directions to explore for generating more diverse simulation data for training more general world models of robot manipulation.

We will update the discussion of future work accordingly.

AuthorsOpenReview ↗

Replay to Reviewer oJJV (3)

W2/Q3: Additional tasks for downstream tasks and include confidence interval.

Part 1- Downstream policy evaluation results with uncertainty.

For downstream policy evaluation in Figure 3, we follow Ctrl-World and evaluate three policies: π0\pi_0, π0\pi_0-FAST, and π0.5\pi_{0.5}. We additionally add 1 more task in rebuttal, and there are a total of 4 tasks, including PnP Banana, PnP Block, Fold Towel, PnP Bread (newly added). We will add the task details to the manuscript.

We now report 95% confidence intervals for all three policy-evaluation metrics, computed over the 12 policy–task points (3 policies ×\times 4 tasks).

Each policy–task pair has 20 real rollouts (10 per exocentric camera), and each world model is rolled out from the same 20 initial states. We use a stratified paired bootstrap with 20,000 resamples: within each (policy, task, camera) stratum we resample the 10 initial states with replacement, and apply the same draw to the real outcomes and to both world models' outcomes. We then recompute Pearson's rr, Spearman's ρ\rho, and MMRV on the resampled success rates. Values are point estimates with 95% bootstrap CIs in brackets.

MetricCtrl-WorldOursΔ\Delta (Ours − Ctrl-World)Resamples where Ours is better
Pearson rr ↑0.196 [−0.090, 0.434]0.390 [0.032, 0.619]+0.194 [−0.133, 0.447]86%
Spearman ρ\rho ↑0.201 [−0.113, 0.438]0.414 [0.005, 0.650]+0.213 [−0.160, 0.488]85%
MMRV ↓0.479 [0.379, 0.604]0.421 [0.308, 0.596]−0.058 [−0.167, 0.079]70%

The success or failure of each episode in the real world and world model is manually labeled after each rollout.

The results show that our model is better on all three metrics. The improvement over Ctrl-World holds in 70–86% of resamples, but the CIs of the paired differences include zero. We therefore describe the improvement as consistent but not statistically significant with 20 rollouts per policy–task pair, and we will revise the paper to state this.

Part 2 - Downstream test-time search (Best-of-N) with uncertainty.

We additionally add two more tasks, leading to a total of 4 tasks in evaluating Best-of-N. Each task has 30 real-robot trials per method (15 per exocentric camera), with base policy being π0.5\pi_{0.5}. We provide the table below giving successes out of 30 (out of 120 overall), the success rate, and an interval. Intervals are 95% credible intervals: each (task, camera, method) group of 15 trials has an independent Jeffreys Beta (1/2, 1/2) prior; we draw 400,000 posterior samples, average the two camera groups for each task (and all groups for the overall row), and report the 2.5th and 97.5th percentiles.

Taskπ\pi-onlyπ\pi + Ctrl-Worldπ\pi + Ours
PnP Banana5/30, 16.7 [7.7, 32.9]6/30, 20.0 [9.7, 37.2]15/30, 50.0 [33.6, 66.4]
PnP Block19/30, 63.3 [45.7, 78.0]17/30, 56.7 [40.2, 71.8]22/30, 73.3 [56.0, 85.6]
PnP Tape7/30, 23.3 [12.0, 40.7]13/30, 43.3 [27.6, 60.7]13/30, 43.3 [27.8, 60.4]
PnP Frog9/30, 30.0 [16.9, 47.7]7/30, 23.3 [12.0, 40.7]13/30, 43.3 [27.5, 60.6]
Overall40/120, 33.3 [27.1, 42.0]43/120, 35.8 [29.3, 44.5]63/120, 52.5 [44.3, 60.4]

Across tasks, search with our model matches or exceeds baselines on all four, while search with Ctrl-World falls below the base policy on two of four. Overall, search with our model improves on the base policy by 19.2 points [6.9, 28.8] and on search with Ctrl-World by 16.7 points [4.4, 26.7] (95% credible intervals from the same posterior samples); both exclude zero.

The success or failure of each episode is manually labeled after each rollout.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Reply to Reviewer U9g2 (1)

We thank the reviewer for the constructive feedback. Please see our responses below.

W1/Q1 Part 1: Continued-training control

We continued training the same Ctrl-World checkpoint on DROID only, with the same learning rate, batch size, and other configurations as Naive Mix (i.e., α=0\alpha=0 vs. 0.9). Due to computational and time limit, we compare the checkpoints every 5k steps up to 35k. Checkpoints for all models are selected by one criterion: the most wins over the pretrained Ctrl-World across the 30 metric cells (3 eval sets ×\times 2 views ×\times 5 metrics), since our expectation is that finetuning can bring generalization to counterfactual cases only seen in simulation while maintaining the performance on near-distribution eval data. This selects DROID-only at 10k (29/30) and Naive Mix (sim:real=90:10) at 35k (26/30).

Eval Set 3 (missed grasps, randomized exocentric cameras, inter-object interactions), selected checkpoints:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoCtrl-World19.770.7340.12438.49600.9
ExoDROID-only (10k)20.25‾\underline{20.25}0.740‾\underline{0.740}0.117‾\underline{0.117}32.35427.9‾\underline{427.9}
ExoNaïve Mix (35k)20.850.7550.10434.15‾\underline{34.15}337.2
WristCtrl-World19.030.6420.30922.09220.8
WristDROID-only (10k)19.33‾\underline{19.33}0.644‾\underline{0.644}0.287‾\underline{0.287}18.84212.0
WristNaïve Mix (35k)19.870.6640.27619.43‾\underline{19.43}213.1‾\underline{213.1}

Eval Set 3, Exo, PSNR$\uparrow$ / FVD$\downarrow$ by training step to show training dynamics:

Model5k10k15k20k25k30k35k
DROID-only20.03 / 486.020.25 / 427.920.02 / 499.120.04 / 501.519.93 / 488.119.90 / 489.819.59 / 655.3
Naïve Mix20.77 / 339.720.90 / 348.720.89 / 353.920.87 / 341.521.12 / 343.121.10 / 336.920.85 / 337.2

First, we can see that sim-and-real co-training improves prediction on Eval Set 3. At the selected checkpoints, Naive Mix is better than DROID-only on PSNR, SSIM, and LPIPS in both views and on Exo FVD (337.2 vs. 427.9). DROID-only has the lower FID in both views, and Wrist FVD is on par (212.0 vs. 213.1).

Second, we observe that, as training steps increase, DROID-only training degrades on Eval Set 3 while co-training does not. In the Exo view, DROID-only peaks at 10k on all five metrics and falls below the pretrained checkpoint by 35k (19.59 PSNR, 655.3 FVD). Naive Mix stays stable through 35k.

Third, it merits attention that co-training actually uses less real-robot data. At α\alpha=0.9, Naive Mix at 35k has seen about 3.5k steps' worth of DROID samples, roughly a third of what DROID-only has seen at 10k.

Fourth, real-only training helps on near-distribution Eval Sets 1 and 2 more than co-training does. On these sets, which are closer to the DROID distribution, DROID-only has lower FID and FVD than Naïve Mix at every step and does not degrade between 5k-35k training steps. Part of the improvement over Ctrl-World reported on these sets is therefore attributable to continued training. We will add DROID-only as a baseline and report those results in Sec.5.2 with updated arguments.

The conclusion from continued training on DROID only vs naive sim-and-real co-training is that: 1. On evaluation sets closer to the training distribution, continued real-only training is as good or better on distribution-level metrics; 2. co-training helps on the conditions it was introduced to cover, off-nominal interactions and novel viewpoints, where continued real-only training degrades.

AuthorsOpenReview ↗

Reply to Reviewer U9g2 (2)

W1/Q1 Part 2: LoRA w/ dynamics consistency loss (DCL) and asymmetric action dropout vs. plain LoRA w/o additional mechanism

We trained a plain LoRA removing both DCL and asymmetric action dropout, while keeping other configurations the same as LoRA w/ DCL. Under the same checkpoint-selection rule (wins over Ctrl-World across 30 metric cells), plain LoRA is selected at 35k with 26/30 wins; LoRA w/ DCL at 35k reaches 30/30.

Eval Set 3, both at 35k:

ViewModelPSNR$\uparrow$SSIM$\uparrow$LPIPS$\downarrow$FID$\downarrow$FVD$\downarrow$
ExoPlain LoRA20.420.7460.11334.23430.0
ExoLoRA w/ DCL20.640.7530.10534.51366.4
WristPlain LoRA19.590.6520.28821.37224.3
WristLoRA w/ DCL19.720.6620.28120.17215.5

On Eval Set 3, adding DCL and dropout improves every metric except Exo FID, with the largest change in Exo FVD (430.0 to 366.4). Plain LoRA's Wrist FVD (224.3) is worse than the pretrained checkpoint (220.8), and with DCL and dropout it is better (215.5).

On Eval Sets 1 and 2, plain LoRA is slightly better than LoRA w/ DCL, since with simulation data, the benefit is concentrated on the off-nominal and novel-viewpoint set.

Finally, we acknowledge that this ablation establishes the combined effect of DCL and asymmetric action dropout at a fixed mixing ratio and adaptation method but does not separate DCL from asymmetric dropout due to computational and time constraints.

Besides, since the main objective of MixWorld is to show that sim-and-real co-training improves models' performance on counterfactual cases unseen in real robot data but covered in simulation data, we leave the study of ratio sweeps and the best co-training strategy to future work. We will update the discussion about future work.

AuthorsOpenReview ↗

Reply to Reviewer U9g2 (3)

W2/Q2 Part 1: The MixWorld variant and checkpoint used for downstream tasks.

We use the MixWorld variant trained with naive mixture (sim:real=90:10) at checkpoint 65k for all downstream tasks. We will add this detail to downstream tasks in Sec. 5.3 and 5.4.

W2/Q2 Part 2: Downstream policy evaluation results with uncertainty.

For downstream policy evaluation in Figure 3, we evaluate three policies: π0\pi_0, π0\pi_0-FAST, and π0.5\pi_{0.5}. The ‘five policies’ in App. A.6 is a typo; Figure 3 uses the three policies above. We will correct it.

We additionally add 1 more task, and there are a total of 4 tasks, including PnP Banana, PnP Block, Fold Towel, PnP Bread (newly added). We will add the task details to the manuscript.

Uncertainty of the rankings. We now report 95% confidence intervals for all three policy-evaluation metrics, computed over the 12 policy–task points (3 policies ×\times 4 tasks).

Each policy–task pair has 20 real rollouts (10 per exocentric camera), and each world model is rolled out from the same 20 initial states. We use a stratified paired bootstrap with 20,000 resamples: within each (policy, task, camera) stratum we resample the 10 initial states with replacement, and apply the same draw to the real outcomes and to both world models' outcomes. We then recompute Pearson's rr, Spearman's ρ\rho, and MMRV on the resampled success rates. Values are point estimates with 95% bootstrap CIs in brackets.

MetricCtrl-WorldOursΔ\Delta (Ours − Ctrl-World)Resamples where Ours is better
Pearson rr ↑0.196 [−0.090, 0.434]0.390 [0.032, 0.619]+0.194 [−0.133, 0.447]86%
Spearman ρ\rho ↑0.201 [−0.113, 0.438]0.414 [0.005, 0.650]+0.213 [−0.160, 0.488]85%
MMRV ↓0.479 [0.379, 0.604]0.421 [0.308, 0.596]−0.058 [−0.167, 0.079]70%

Per-policy/task success counts (policy evaluation).

TaskPolicyRealCtrl-WorldOurs
PnP blockπ0\pi_03/2011/2010/20
PnP blockπ0\pi_0-FAST17/2010/2010/20
PnP blockπ0.5\pi_{0.5}13/207/208/20
PnP bananaπ0\pi_05/2011/2012/20
PnP bananaπ0\pi_0-FAST8/206/209/20
PnP bananaπ0.5\pi_{0.5}10/206/2010/20
Fold towelπ0\pi_01/204/205/20
Fold towelπ0\pi_0-FAST14/2017/2016/20
Fold towelπ0.5\pi_{0.5}15/206/2013/20
PnP breadπ0\pi_03/200/203/20
PnP breadπ0\pi_0-FAST14/201/207/20
PnP breadπ0.5\pi_{0.5}10/200/201/20
Total113/24079/240104/240

The success or failure of each episode in the real world and world model is manually labeled after each rollout.

The results show that our model is better on all three metrics. The improvement over Ctrl-World holds in 70–86% of resamples, but the CIs of the paired differences include zero. We therefore describe the improvement as consistent but not statistically significant with 20 rollouts per policy–task pair, and we will revise the paper to state this.

We will add the updated policy-evaluation results and figure in the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.