acceptodds
Under review as a conference paper at ICLR 2027

OpenWorld: Scaling World Model Pretraining

Abstract

World models trained on large-scale data hold immense promise for emulating complex interactions with the physical world. While action-conditioned video generation as a world model has gained traction due to its potential to scale, the underlying scaling behaviors of these models remain poorly understood. Most existing works rely on finetuning fixed-size pretrained text-to-video models on action data; however, a systematic understanding of how model size, data, and compute affect action-conditioned video world models is lacking. To close this gap, we curate an action-video dataset comprising 4 million video clips (120 billion image tokens) spanning 10,000 hours of human, robot, and navigation data. We then introduce a fully open recipe, OpenWorld, that enables scaling models from 600M to 7.5B parameters. In a dedicated isoFLOP sweep matched to our deployment-scale training setup, we find that held-out diffusion loss exhibits a strongly model-heavy compute frontier, with . Separately, a constant-learning-rate training-loss frontier fitted on models up to 1.02B parameters predicts the excluded 7.5B run to within 2.5% near its predicted optimal compute. Rollout evaluations show that larger deployment-scale checkpoints improve in-domain video fidelity, downstream finetuning, and behavior-level trajectory matching, while also revealing that standard paired video metrics are not always reliable proxies for action following. Finally, we demonstrate the utility of OpenWorld across downstream applications, including real-time human-interaction simulation, multi-view robot execution simulation across diverse robot morphologies, and building a robot leaderboard for evaluating policies in the cloud. Qualitative rollouts from OpenWorld-7.5B further show plausible gravity, contact-rich interaction, and deformable-object dynamics.

Primary area:
foundation or frontier models, including LLMs
Keywords:
world models, scaling laws, video generation, embodied AI, action-conditioned video modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cMNCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents OpenWorld: an embodied robotics world-model. In addition, this paper studies the scaling properties of world models trained from scratch on video-action data.

Critical strengths
  • The presentation is reasonable and easy to follow.
Critical weaknesses
  • It is unclear whether the core motivation for the paper is a controlled study of scaling dynamics for world models or to present a new model. This results in both being done quite poorly.
  • The scaling results are poorly constructed and don't make sense:
    • The paper claims that action-conditioned diffusion models scale exetremely poorly with increased data. This is very unintuitive and contradicts previous work [1]. Much more likely this result is observed due to overfitting on the data (especailly the smaller subsets that were combined to make the corpera) due to the re-sampling technique.
    • The learning rate cannot be fixed across model sizes, as this results in sub-optimal models at different sizes, and poor resulting fit.
    • For the purpose of understanding scaling trends, sweep B is mysterious as it has dramatically different settings than sweep A and the 7.5B model.
    • The extropolation to 7.5B is deeply troubled: firstly, the 7.5B model is trained on a different data distribution. Rendering it useless for validating any scaling trend. Furthemore, even if the 7.5B model was trained on the same data distribution, a single model only confirms closeness the fit curve, not whether it is optimal.
    • Other non-standard choices such as the trapezoidal learning rate schedule do not help.
  • OpenWorld as independent model:
    • Another interesting direction would be comparing OpenWorld---A model trained end-to-end on video-action data---to the more common method of adapting video generation models into world models. But OpenWorld is not compared to any baselines. Section 4 just seems to be generic applications for any embodied robotics world model and does not compare against any baselines.

[1] Z. Liang, H. He, C. Yang, and B. Dai, “Scaling Laws For Diffusion Transformers,” Mar. 2026, doi: 10.48550/arxiv.2410.08184.

Questions for authors

None

3 comments

Response to Reviewer cMNC

We thank the reviewer for the feedback. Our central contribution is a controlled study of how to allocate compute when pretraining action-conditioned video world models. We clarify what the allocation law measures, why Sweep B uses a first-epoch setup, and what the 7.5B run tells us about training-loss prediction.

 

[W1] Clarification of our contribution

The reviewer asks whether the paper primarily studies scaling or introduces a new model. The question posed in Section 1 and tested in Section 3 is how to allocate pretraining compute between model size and training tokens within a Diffusion Transformer family. OpenWorld provides the pretraining recipe, corpus, and checkpoints used to study that question. Section 4 uses those checkpoints to illustrate potential applications.

 

[W2] Allocation between model size and training data

The reviewer interprets the model-heavy result as a claim that adding data barely improves action-conditioned diffusion models. Figure 2 measures how the loss-minimizing allocation changes as the compute budget increases. In Sweep A, the fitted model size grows as Nopt∝C0.94N_{\mathrm{opt}} \propto C^{0.94} and the fitted token budget grows as Dopt∝C0.06D_{\mathrm{opt}} \propto C^{0.06}. Both increase with compute, with model size growing much faster. The gray curves in Figure 3 also show training loss decreasing as individual models process more tokens. The small exponent on DoptD_{\mathrm{opt}} describes how the allocated token budget grows along the frontier. Measuring the effect of adding unique data at a fixed model size is a different experiment.

Liang et al. [1] estimate the model–data allocation for text-to-image generation on image–text pairs. We estimate it for action-conditioned video prediction, where the model predicts future frames given past observations and actions. The fitted exponents describe the trade-off measured in each setting. Applying the text-to-image exponents to action-conditioned video would require evidence that this trade-off transfers across settings. Liang et al. explicitly leave transfer to video and other modalities open in Appendix I [1].

 

[W2] First-epoch robustness in Sweep B

The reviewer asks what Sweep B establishes given its different training settings. Sweep A is the primary isoFLOP study and uses weighted sampling, which can revisit clips. Sweep B asks whether the model-heavy trend also appears in a first pass through non-overlapping clips. We fit Sweep B independently using annealed endpoints from the first epoch. Its model-size exponent is 0.89±0.030.89 \pm 0.03, compared with 0.94±0.020.94 \pm 0.02 in Sweep A (Appendix C.3). The model-heavy trend therefore also appears without revisiting clips in Sweep B.

Because batch size and clip length change as well as sampling, Sweep B provides a robustness check for the direction of the trend. Prior scaling studies likewise examine different exposure regimes, including mostly single-pass training in Liang et al. [1] and repeated data in Muennighoff et al. [2].

 

[W2] Shared learning rate across model sizes

The reviewer raises the concern that the best learning rate may vary with model size. We tuned the base learning rate once and used the selected rate of 10−410^{-4} with a 10,000-step warmup across Sweep A. This keeps the training recipe consistent as we vary model size and training tokens at each FLOP budget.

In the study cited in the review, Liang et al. also use a shared learning rate of 10−410^{-4} across model sizes in their DiT isoFLOP experiments [1]. They discuss in Appendix I that tuning hyperparameters more carefully for each configuration could improve the accuracy of compute-optimal predictions. Therefore, we leave more fine-grained hyperparameter tuning across model sizes as an important direction for future work.

 

[W2] Cooldown schedule and 7.5B extrapolation

We use a cooldown to compare held-out loss at each target FLOP budget. We reduce the learning rate tenfold by cosine decay over the final 10% of steps, then evaluate the resulting checkpoint (Section 3.2). This follows the constant-learning-rate-plus-cooldown approach used by Hägele et al. for scaling studies [3].

Figure 3 uses constant-learning-rate training histories for both the fit on smaller Sweep A models and the excluded 7.5B run. The smoothed 7.5B loss is 2.48% above the predicted frontier at 3.12 × 10²¹ FLOPs and 2.72% above it at the predicted budget of 3.83 × 10²¹ FLOPs. All runs in the Figure 2 isoFLOP sweep use the same training mixture. The separate 7.5B run in Figure 3 uses the same underlying data sources as Sweep A, with different AgiBotWorld variants covering the same 35 tasks. This distinction does not change our conclusion: the frontier fitted to smaller Sweep A models predicts the excluded 7.5B run’s training loss to within 2.72% at both reported compute budgets.

[W3] Comparison with adapted video generators

The reviewer suggests comparing OpenWorld with world models adapted from pretrained video generators. Such a comparison would answer which approach performs better on a given downstream task. Our study asks how to allocate compute during pretraining. Starting from a fixed video checkpoint fixes the backbone size and its prior training history, while our sweep varies model size and token exposure together under matched FLOP budgets. Comparing different model architectures or identifying the best architecture across families would address separate research questions.

 

References

[1] Liang et al. Scaling Laws for Diffusion Transformers. ICLR 2026, arXiv:2410.08184.

[2] Muennighoff et al. Scaling Data-Constrained Language Models. NeurIPS 2023, arXiv:2305.16264.

[3] Hägele et al. Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations. NeurIPS 2024, arXiv:2405.18392.

Reviewer_cMNCOpenReview ↗

Reviewer response

The question posed in Section 1 and tested in Section 3 is how to allocate pretraining compute between model size and training tokens within a Diffusion Transformer family.

The reviewers concerns that the study of how to allocate compute is flawed still remain. Especially the scaled up model having a different data distribution.

Learning rate settings in Liang et al.

A common flaw does not make it not a flaw. [1] would be a better reference for scaling experiments.

An aside on the dava vs. model params scaling ratio

Although action-conditioned video diffusion is different than standard video diffusion or LLM scaling. The scaling ratio being dramatically different than video diffusion (which itself is not that different from LLM scaling---a much more different domain) is very counterintuitive (very interesting if true!) But this paper has not provided nearly a controlled enough study / experimentation to rigorously justify that claim.


[1] J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” Mar. 2022, doi: 10.48550/arxiv.2203.15556.

Reviewer_im8LOpenReview ↗
Rating
3
Confidence
2(−1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

OpenWorld pretrains action-conditioned video world models from 600M to 7.5B parameters on approximately 10,000 hours of heterogeneous interaction data. Scaling sweeps identify a model-heavy diffusion-loss frontier, while larger deployment models improve rollout fidelity, downstream adaptation, and trajectory matching.

Critical strengths
  • Two systematic isoFLOP sweeps provide valuable evidence about compute allocation for action-conditioned video pretraining. The distinction between held-out-loss scaling and training-loss extrapolation makes the findings easier to assess.

  • The large heterogeneous corpus, supplied training code, and evaluations across model sizes provide a useful foundation for further research. Human trajectory judgments also support the broad improvement observed at 7.5B.

Critical weaknesses
  • The 7.5B extrapolation compares a single training-loss trajectory against a predicted frontier, with different data preprocessing. Agreement in loss does not validate compute-optimal allocation at that scale.

  • AutoEval covers only four tasks and lacks incorrect or shuffled-action controls to separate action responsiveness from visual-quality improvements. For 7.5B successful trajectories, VLM matching is 51.7% versus 25.0% under human evaluation, limiting confidence in absolute behavioral scores.

  • The benefit of heterogeneous pretraining is not isolated by the corpus-fraction and pretraining-versus-scratch comparisons. A matched robot-only comparison is needed to establish whether human and navigation data improve robot dynamics or transfer.

Questions for authors

Please refer to the critical weaknesses above.

3 comments

Response to Reviewer im8L

We thank the reviewer for recognizing the value of the systematic sweeps, open corpus, and training pipeline. We clarify what the 7.5B run tells us about loss prediction, what the human judgments show on AutoEval, and why the scaling study holds the data mixture fixed.

 

7.5B training-loss extrapolation

The reviewer asks how to interpret the agreement between the 7.5B run and the predicted loss frontier. We fit the training-loss frontier on smaller Sweep A models and use the excluded 7.5B run to check the extrapolation. Its smoothed training loss is 2.48% above the predicted frontier at 3.12 × 10²¹ FLOPs and 2.72% above it at the predicted budget of 3.83 × 10²¹ FLOPs (Appendix C.4).

All runs in the Figure 2 isoFLOP sweep use the same training mixture. The separate 7.5B run in Figure 3 uses the same underlying data sources as Sweep A, with different AgiBotWorld variants covering the same 35 tasks. This distinction does not change our conclusion: the frontier fitted to smaller Sweep A models predicts the excluded 7.5B run’s training loss to within 2.72% at both reported compute budgets.

 

AutoEval coverage and human judgments

The reviewer raises concerns about task coverage, action controls, and the gap between VLM and human judgments. We evaluate all four tasks in AutoEval’s public drawer and sink setup [1], using 479 paired trajectories. Each generated rollout uses the initial frame and recorded actions of its real counterpart. The matching rubric checks gripper motion, contact, object state changes, and final outcomes, including whether the rollout reproduces a failed execution. Shuffling the actions would test how the generated behavior changes when the commands change.

On the same 60 successful real executions at 7.5B, the VLM judges 51.7% of the rollouts as matches, compared with 25.0% for humans (Table 11). The human results still show an improvement on both subsets. The 7.5B checkpoint matches 25% of successful executions and 45% of failed executions, compared with 0% and 1.67%, respectively, for each smaller checkpoint (Table 9).

 

Data mixture in the isoFLOP study

The reviewer asks whether adding human and navigation data improves robot modeling at equal compute.Table 4 compares corpus fractions at fixed model size, and Table 5 compares pretrained initialization with training from scratch for Bridge V2 adaptation. Our main contribution is an isoFLOP scaling law for a fixed data mixture. In Sweep A, we hold that mixture fixed and vary model size and training-token exposure at each FLOP budget to find the allocation that minimizes held-out diffusion loss. A matched robot-only comparison would answer a separate question about which data to include, which is outside the scope of this scaling study.

 

References

[1] Zhou et al. AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World. CoRL 2025, arXiv:2503.24278.

Thank you for the detailed response. Although some of my concerns, such as validating the compute-optimal allocation at the 7.5B scale and the shuffled-action control, have not been fully addressed, given that I am not a specialist in this line of work and my confidence is moderate, I will maintain my original positive score.

Thank you for taking the time to consider our response. We appreciate your constructive feedback, which helped us clarify our paper’s contributions and limitations.

Reviewer_ix3QOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates scaling action-conditioned world model pretraining. It uses 400M clips covering robot manipulation, human interaction, and navigation tasks. The authors run isoFLOP experiments and find that allocating more compute to the model can get better held-out valid loss. They also fit the training loss curve to extrapolate performance for a separate 7.5B-parameter run that was not fully trained. Besides, the pretrained models with larger parameters produce higher-fidelity videos and better trajectory matching as judged by humans and VLMs.

Critical strengths
  • Detailed scaling analysis. This paper compares models of different sizes under fixed compute budgets with annealed endpoints. A separate first-epoch sweep also supports this model-heavy trend. This setup gives more meaningful results than just comparing a sequence of larger models.
  • Useful evaluation insights. Tables 9–11 show that standard pixel-level metrics can conflict with whether predictions match real behavior. Human evaluation backs up the observed scaling trend and highlights weaknesses in VLM-based evaluators.
  • Strong engineering and resource contributions. The diverse action-video dataset, released training pipeline, model family, and documented stabilization tricks create a solid base for future work.
Critical weaknesses
  • Fine-grained action following lacks sufficient evaluation. Figure 4 and Tables 9–11 use human and VLM ratings to judge overall trajectory matching, but they do not directly measure robot trajectory alignment, response scale, or timing accuracy. As a result, the evaluation cannot clearly tell apart better visual quality and rough interaction consistency from precise adherence to input actions.
  • The work shows functional behavior but does not fully validate it. Figure 5 and Table 12 report policy performance within OpenWorld itself, without checking whether these results match performance in a reference environment. For example, Table 12 ranks OpenVLA above π₀.₅, whereas some external evaluations report the opposite ordering. Although differences in tasks and policy adaptation may explain this discrepancy, the absence of matched reference-environment evaluation makes it difficult to determine whether the ranking reflects policy capability or policy-dependent simulator bias.
  • Loss-based scaling results are not linked to compute-optimal behavioral performance. Figure 2 finds the compute-optimal split using diffusion loss, but there is no evidence that this split also yields the best action-following performance. Figure 4 shows that larger models trained with more compute achieve better behavioral matching, but does not establish whether increasing model size is the best use of a fixed compute budget for improving action fidelity. The result is suggestive, but the experiment may not be fully controlled.
  • It remains unclear how much the mixed dataset contributes. Current results do not isolate whether human and navigation data improve robot modeling compared to robot-only training with the same compute budget.
Questions for authors
  • Can the authors add fine-grained action-following metrics alongside trajectory-level ratings, such as tracked end-effector error? Controlled action swaps or perturbations would also help verify whether the model responds properly to conditioning actions.
  • How well do policy rankings and success rates from this simulator match a reference simulator or real-world setup, using identical tasks and starting states? Alternatively, can generated rollouts boost policy performance in that reference environment, following evaluation frameworks like WorldArena?
  • Does the model-heavy compute allocation still look favorable when measured by action-following accuracy under fixed compute, instead of diffusion loss alone? How sensitive is the 7.5B extrapolation to fitting choices and the difference in preprocessing?
  • With equal training budgets, does mixing human and navigation data yield better robot prediction than training purely on robot data?
4 comments

Response to Reviewer ix3Q:

We thank the reviewer for recognizing the controlled scaling analysis, the evaluation insights and the open resources. The response below clarifies evidence for using trajectory-matching, what the loss frontier establishes, and the scope of extrapolation, policy-ranking and dataset comparisons. We will make these distinctions explicit in the revision.

 

[Q1, W1]: Trajectory matching and action fidelity

The reviewer requests finer measures of action fidelity and controls that separate action responsiveness from visual quality. Our trajectory matching evaluates whether OpenWorld reproduces the task-relevant effects of the recorded actions. For each held-out AutoEval trajectory, we initialize OpenWorld with the first frame, replay the recorded action sequence, and compare the generated video with the corresponding real execution. The rubric checks agreement in gripper motion, gripper–object contact, object state changes, and final gripper and object states (Appendix D.3). For example, if the real gripper misses the object, a generated rollout that successfully picks it up is a mismatch. The score therefore captures behavioral fidelity at the level of task-relevant interactions across both successful and failed executions.

Moreover, task-level assessment in generated rollouts is a common criterion for evaluating robot policies in learned world models, as used in WorldGym, Ctrl-World, Veo (Robotics), WorldEval, and dWorldEval [1–5]. World-Gymnast also uses task completion in generated rollouts as a reward for policy training [6].

 

[Q2, W2]: Policy evaluation on Bridge

The reviewer asks whether the policy rankings in OpenWorld agree with real-robot results, particularly for OpenVLA and π₀.₅. Figure 5 and Table 12 illustrate a potential application of OpenWorld as a testbed for policy evaluation. We run seven policy checkpoints in closed loop on Bridge tasks, where each policy acts on the observations generated by OpenWorld, and score task completion in the resulting rollouts.

Finding a directly comparable real-robot reference is difficult because the comparison needs to match the policy checkpoints, their fine-tuning, the tasks, and the starting states. Two published Bridge/WidowX real-robot evaluations rank OpenVLA above Octo, consistent with Table 12 [8, 9]. For the pair highlighted by the reviewer, OpenVLA scores higher than our Bridge-finetuned π₀.₅ checkpoint. Shi et al. report the reverse ordering for checkpoints fine-tuned on four Franka tasks [10]. The real-robot ordering of these two checkpoints on the Table 12 tasks has not yet been established. Policy performance can vary across checkpoints, tasks, and environments. OpenWorld offers a way to compare candidate policies on the tasks of interest before running real-robot trials.

[Q3, W3]: Loss-based compute allocation and action following

The reviewer asks whether the allocation that minimizes diffusion loss also gives the best action following at the same training budget. In Figure 2, we fit the scaling law using held-out diffusion loss. It describes how loss changes with model size, training tokens, and FLOPs, and estimates the allocation that minimizes loss at each budget. We first want to see that adding compute improves the pretraining objective. We then ask whether the benefits of scaling also show up in downstream behavior. This is the same approach taken in Chinchilla, which uses pretraining loss to fit compute allocation and measures downstream performance separately [7].

For this downstream check, we measure trajectory matching on four out-of-distribution AutoEval tasks. We generate each rollout from the real trajectory’s initial frame and recorded actions, then compare it with the corresponding real execution. Figure 4 shows that larger models trained with more compute match the real trajectories more often. Which allocation gives the best action following at fixed compute remains an open question.

7.5B training-loss extrapolation

The reviewer also asks how sensitive the 7.5B extrapolation is to fitting choices and preprocessing. Appendix C.4 fits the training-loss frontier on smaller Sweep A runs, with EMA smoothing at decay 0.85 and quadratic fits in log model size at eight compute budgets. The 7.5B run is excluded from the fit. Its smoothed loss is 2.48% above the predicted frontier at 3.12 × 10²¹ FLOPs, and the interpolated gap is 2.72% at the predicted budget of 3.83 × 10²¹ FLOPs.

All runs in the Figure 2 isoFLOP sweep use the same training mixture. The separate 7.5B run in Figure 3 uses the same underlying data sources as Sweep A, with different AgiBotWorld variants covering the same 35 tasks. This distinction does not change our conclusion: the frontier fitted to smaller Sweep A models predicts the excluded 7.5B run’s training loss to within 2.72% at both reported compute budgets.

 

[Q4, W4]: Data mixture in the isoFLOP study

The reviewer asks how adding human and navigation data affects robot modeling under a matched training budget. Our contribution is an isoFLOP scaling law for allocating compute on a fixed data mixture. In Sweep A, we hold the mixture fixed and vary model size and training-token exposure at each FLOP budget to find the allocation that minimizes held-out diffusion loss. A matched robot-only comparison would address a separate meaningful research question about what data to include, which is out of this paper's scope.

 

References

[1] Quevedo et al. WorldGym: World Model as An Environment for Policy Evaluation. ICLR 2026.

[2] Guo et al. Ctrl-World: A Controllable Generative World Model for Robot Manipulation. ICLR 2026.

[3] Gemini Robotics Team, Google DeepMind. Evaluating Gemini Robotics Policies in a Veo World Simulator. arXiv:2512.10675, 2025.

[4] Li et al. WorldEval: World Model as Real-World Robot Policies Evaluator. arXiv:2505.19017, 2025.

[5] Li et al. dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model. arXiv:2604.22152, 2026.

[6] Sharma et al. World-Gymnast: Training Robots with Reinforcement Learning in a World Model. arXiv:2602.02454, 2026.

[7] Hoffmann et al. Training Compute-Optimal Large Language Models. NeurIPS 2022, arXiv:2203.15556.

[8] Kim et al. OpenVLA: An Open-Source Vision-Language-Action Model. arXiv:2406.09246, 2024.

[9] Qu et al. SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Models. RSS 2025.

[10] Shi et al. Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models. arXiv:2602.12628, 2026. Table 1.

Thank you for the response. I still have a few concerns:

  • W1 — Action fidelity. While I agree that the "trajectory matching" evaluation criterion considers more than just task success—encompassing gripper motion, contact events, and changes in object states—the final binary "match/mismatch" label still fails to capture the degree to which the generated actions remain faithful to the input actions.

  • W2 — Policy ranking. In Table 12, OpenVLA is ranked higher than π₀.₅, which is puzzling given that previous evaluations have shown the opposite ranking. I am concerned that OpenWorld might simulate the consequences of one policy more accurately than the other, potentially biasing the rankings.

  • W3 — Scaling and action fidelity. The results for the 7.5B-parameter model in Figure 3 are encouraging; however, the preprocessing method used for the AgiBot data in this experiment differs from that used for the smaller-scale models when fitting the performance frontier. Consequently, it is difficult to determine to what extent the observed consistency can be attributed to the scaling trend itself.

Response to Reviewer ix3Q:

We thank the reviewer for the thoughtful follow-up.

[Q1, W1]: Fine-grained Action Fidelity

Our AutoEval evaluation asks whether larger pretrained checkpoints more often reproduce the task-relevant behavior of a recorded execution when conditioned on the same initial frame and actions. The binary match rate measures how often this happens. We agree that it leaves the magnitude and timing of individual motion deviations unresolved. Quantifying these deviations with metrics such as end-effector displacement error and contact timing is a valuable direction for future work in better benchmarking video world model's downstream task performance.

 

[Q2, W2] Closed-loop policy evaluation as an application:

Figure 5 and Table 12 illustrate how video world models could be used to evaluate policies by running them in closed loop and comparing their task success rates. OpenWorld already captures some signal about relative policy performance. For example, it ranks OpenVLA above Octo on Bridge tasks, consistent with published WidowX real-robot evaluations [8, 9].

For these comparisons to be reliable, the world model needs to reproduce the consequences of each policy’s actions. Figure 4 shows that larger models trained with more compute match recorded trajectories more often, suggesting that further scaling may help improve rollout fidelity. For OpenVLA and π₀.₅ specifically, we agree with the reviewer that OpenWorld may simulate one policy more faithfully than the other, which could affect their ordering in Table 12.

 

[Q3, W3]: Data Processing

For the 7.5B run, we filtered out lower-quality AgiBotWorld data. Both the 7.5B run and the smaller models used data from the same sources and covered the same 35 AgiBotWorld tasks. Given this, we do not expect the filtering to change our qualitative scaling conclusion, although it may affect the exact gap between the 7.5B training loss and the predicted frontier in Figure 3.

Comments on the paper

1 comment

Response to All Reviewers:

We thank the reviewers for their feedback. We want to clarify that our main contribution is a controlled study of how to allocate pretraining compute between model size and training tokens for action-conditioned video world models. For the DiT family and corpus we study, both the model size and the token budget that minimize held-out diffusion loss grow with compute, with model size growing faster. In the revision, we will clarify this finding in the abstract and introduction, explain how the first-epoch Sweep B checks the trend, and describe the shared learning rate, cooldown schedule, and training-loss extrapolation. training-loss extrapolation. We will also highlight the human trajectory-matching results on AutoEval and present the Bridge rollouts as an example of how OpenWorld could support closed-loop policy evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.