- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper presents OpenWorld: an embodied robotics world-model. In addition, this paper studies the scaling properties of world models trained from scratch on video-action data.
Critical strengths
- The presentation is reasonable and easy to follow.
Critical weaknesses
- It is unclear whether the core motivation for the paper is a controlled study of scaling dynamics for world models or to present a new model. This results in both being done quite poorly.
- The scaling results are poorly constructed and don't make sense:
- The paper claims that action-conditioned diffusion models scale exetremely poorly with increased data. This is very unintuitive and contradicts previous work [1]. Much more likely this result is observed due to overfitting on the data (especailly the smaller subsets that were combined to make the corpera) due to the re-sampling technique.
- The learning rate cannot be fixed across model sizes, as this results in sub-optimal models at different sizes, and poor resulting fit.
- For the purpose of understanding scaling trends, sweep B is mysterious as it has dramatically different settings than sweep A and the 7.5B model.
- The extropolation to 7.5B is deeply troubled: firstly, the 7.5B model is trained on a different data distribution. Rendering it useless for validating any scaling trend. Furthemore, even if the 7.5B model was trained on the same data distribution, a single model only confirms closeness the fit curve, not whether it is optimal.
- Other non-standard choices such as the trapezoidal learning rate schedule do not help.
- OpenWorld as independent model:
- Another interesting direction would be comparing OpenWorld---A model trained end-to-end on video-action data---to the more common method of adapting video generation models into world models. But OpenWorld is not compared to any baselines. Section 4 just seems to be generic applications for any embodied robotics world model and does not compare against any baselines.
[1] Z. Liang, H. He, C. Yang, and B. Dai, “Scaling Laws For Diffusion Transformers,” Mar. 2026, doi: 10.48550/arxiv.2410.08184.
Questions for authors
None
3 comments
[W3] Comparison with adapted video generators
The reviewer suggests comparing OpenWorld with world models adapted from pretrained video generators. Such a comparison would answer which approach performs better on a given downstream task. Our study asks how to allocate compute during pretraining. Starting from a fixed video checkpoint fixes the backbone size and its prior training history, while our sweep varies model size and token exposure together under matched FLOP budgets. Comparing different model architectures or identifying the best architecture across families would address separate research questions.
References
[1] Liang et al. Scaling Laws for Diffusion Transformers. ICLR 2026, arXiv:2410.08184.
[2] Muennighoff et al. Scaling Data-Constrained Language Models. NeurIPS 2023, arXiv:2305.16264.
[3] Hägele et al. Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations. NeurIPS 2024, arXiv:2405.18392.
Reviewer response
The question posed in Section 1 and tested in Section 3 is how to allocate pretraining compute between model size and training tokens within a Diffusion Transformer family.
The reviewers concerns that the study of how to allocate compute is flawed still remain. Especially the scaled up model having a different data distribution.
Learning rate settings in Liang et al.
A common flaw does not make it not a flaw. [1] would be a better reference for scaling experiments.
An aside on the dava vs. model params scaling ratio
Although action-conditioned video diffusion is different than standard video diffusion or LLM scaling. The scaling ratio being dramatically different than video diffusion (which itself is not that different from LLM scaling---a much more different domain) is very counterintuitive (very interesting if true!) But this paper has not provided nearly a controlled enough study / experimentation to rigorously justify that claim.
[1] J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” Mar. 2022, doi: 10.48550/arxiv.2203.15556.
Response to Reviewer cMNC
We thank the reviewer for the feedback. Our central contribution is a controlled study of how to allocate compute when pretraining action-conditioned video world models. We clarify what the allocation law measures, why Sweep B uses a first-epoch setup, and what the 7.5B run tells us about training-loss prediction.
[W1] Clarification of our contribution
The reviewer asks whether the paper primarily studies scaling or introduces a new model. The question posed in Section 1 and tested in Section 3 is how to allocate pretraining compute between model size and training tokens within a Diffusion Transformer family. OpenWorld provides the pretraining recipe, corpus, and checkpoints used to study that question. Section 4 uses those checkpoints to illustrate potential applications.
[W2] Allocation between model size and training data
The reviewer interprets the model-heavy result as a claim that adding data barely improves action-conditioned diffusion models. Figure 2 measures how the loss-minimizing allocation changes as the compute budget increases. In Sweep A, the fitted model size grows as Nopt∝C0.94 and the fitted token budget grows as Dopt∝C0.06. Both increase with compute, with model size growing much faster. The gray curves in Figure 3 also show training loss decreasing as individual models process more tokens. The small exponent on Dopt describes how the allocated token budget grows along the frontier. Measuring the effect of adding unique data at a fixed model size is a different experiment.
Liang et al. [1] estimate the model–data allocation for text-to-image generation on image–text pairs. We estimate it for action-conditioned video prediction, where the model predicts future frames given past observations and actions. The fitted exponents describe the trade-off measured in each setting. Applying the text-to-image exponents to action-conditioned video would require evidence that this trade-off transfers across settings. Liang et al. explicitly leave transfer to video and other modalities open in Appendix I [1].
[W2] First-epoch robustness in Sweep B
The reviewer asks what Sweep B establishes given its different training settings. Sweep A is the primary isoFLOP study and uses weighted sampling, which can revisit clips. Sweep B asks whether the model-heavy trend also appears in a first pass through non-overlapping clips. We fit Sweep B independently using annealed endpoints from the first epoch. Its model-size exponent is 0.89±0.03, compared with 0.94±0.02 in Sweep A (Appendix C.3). The model-heavy trend therefore also appears without revisiting clips in Sweep B.
Because batch size and clip length change as well as sampling, Sweep B provides a robustness check for the direction of the trend. Prior scaling studies likewise examine different exposure regimes, including mostly single-pass training in Liang et al. [1] and repeated data in Muennighoff et al. [2].
[W2] Shared learning rate across model sizes
The reviewer raises the concern that the best learning rate may vary with model size. We tuned the base learning rate once and used the selected rate of 10−4 with a 10,000-step warmup across Sweep A. This keeps the training recipe consistent as we vary model size and training tokens at each FLOP budget.
In the study cited in the review, Liang et al. also use a shared learning rate of 10−4 across model sizes in their DiT isoFLOP experiments [1]. They discuss in Appendix I that tuning hyperparameters more carefully for each configuration could improve the accuracy of compute-optimal predictions. Therefore, we leave more fine-grained hyperparameter tuning across model sizes as an important direction for future work.
[W2] Cooldown schedule and 7.5B extrapolation
We use a cooldown to compare held-out loss at each target FLOP budget. We reduce the learning rate tenfold by cosine decay over the final 10% of steps, then evaluate the resulting checkpoint (Section 3.2). This follows the constant-learning-rate-plus-cooldown approach used by Hägele et al. for scaling studies [3].
Figure 3 uses constant-learning-rate training histories for both the fit on smaller Sweep A models and the excluded 7.5B run. The smoothed 7.5B loss is 2.48% above the predicted frontier at 3.12 × 10²¹ FLOPs and 2.72% above it at the predicted budget of 3.83 × 10²¹ FLOPs. All runs in the Figure 2 isoFLOP sweep use the same training mixture. The separate 7.5B run in Figure 3 uses the same underlying data sources as Sweep A, with different AgiBotWorld variants covering the same 35 tasks. This distinction does not change our conclusion: the frontier fitted to smaller Sweep A models predicts the excluded 7.5B run’s training loss to within 2.72% at both reported compute budgets.