acceptodds
Withdrawn from ICLR 2027

ScaleWorld: Streaming High-Resolution World Generation through Shared Prediction and Refinement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High-resolution generation is crucial for camera-controlled world models, yet direct high-resolution rollout is costly and often degrades visual details. We present , a causal video model that learns world prediction and high-resolution refinement in shared parameters. This allows the model to refine its own predictions for improved visual detail while reducing the cost of high-resolution generation. Specifically, each low-resolution chunk is completed through four-step camera-controlled prediction and then conditions a second invocation of the same backbone for one-step refinement. Realizing this process requires addressing two core challenges: (i) Efficiency. Separate causal states maintain a long prediction context at low resolution and a short refinement context at high resolution, reducing context cost. (ii)Capability integration. We separate specialist optimization from consolidation and introduce task-asymmetric on-policy distillation (TA-OPD), which transfers prediction and refinement behaviors into a shared student under their respective sampling and history policies. Extensive experiments show that effectively unifies prediction and refinement while reducing the context-storage cost of high-resolution world generation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Streaming Video Generation, World Model, High Resolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nKMBOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ScaleWorld, a self-cascading causal video model that integrates camera-controlled low-resolution world prediction and high-resolution conditional refinement into one shared transformer backbone. It proposes Task-Asymmetric On-Policy Distillation (TA-OPD) to resolve conflicting gradients during multi-task distillation by first training task-specific specialists and then consolidating them into a shared student model via local velocity matching. Experiments on the WBench benchmark show the unified model achieves comparable generation quality to a separate two-specialist cascade, while reducing peak GPU memory by ~21% and retaining native prediction capability.

Critical strengths
  1. Proposes TA-OPD to effectively address gradient conflicts in joint prediction-refinement distillation, delivering a solid algorithmic solution for single-backbone self-cascaded world generation.
  2. Provides rigorous analytical and empirical analysis of deployment efficiency (memory, latency, throughput), offering valuable engineering insights for real-time streaming systems.
Critical weaknesses
  1. Insufficient motivation for single-backbone unification: WM and SR are inherently sequential execution stages with no fundamental requirement to share one set of model weights. The observed "conflicting gradient directions and imbalanced gradient magnitudes" actually reflect inherent optimization tensions between the two tasks. TA-OPD only strikes a compromise rather than achieving optimal performance on both tasks, and the unified model even regresses on multiple metrics compared with separate specialist models (as shown by comparisons in Table 3 & 4). The paper fails to deliver a strong justification for why full weight sharing is necessary or uniquely advantageous.
  2. Unutilized efficiency gains with no capability breakthrough: Weight sharing only reduces peak GPU memory, but the saved memory and compute headroom are not leveraged to push the limits of generation capabilities (such as higher resolution, longer temporal horizon, or lower inference latency). The work does not achieve improvements in actual generation quality or capability boundaries, so the efficiency gain does not translate into a meaningful advance in world modeling performance.
  3. Incomplete introduction narrative: The introduction ends abruptly after the description of TA-OPD, lacking a concluding paragraph that summarizes the full scope of the work and highlights core contributions.
Questions for authors
  1. Could you add sample generated video clips as supplementary material to allow visual evaluation of temporal stability, detail fidelity and flickering artifacts?
  2. Have you tried leveraging the reduced memory footprint to push generation limits (higher resolution, longer horizon, faster throughput) instead of only matching existing quality?
  3. Have you compared full weight sharing with lighter parameter reuse strategies (e.g., shared bottom layers + task-specific heads, LoRA adapters) to map the full quality-efficiency tradeoff frontier?
Reviewer_g1vxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the generation efficiency issue in world models by employing a coarse-to-fine generation approach. It also introduces the TA-OPD method, which enables a single model to perform both prediction and super-resolution tasks.

Critical strengths
  1. The paper presents a well-defined problem and clear conclusions.
  2. The methodology is sound, and the experimental evaluation is comprehensive.
Critical weaknesses
  1. Much of the contribution consists of integrating existing engineering techniques.
  2. The paper lacks an in-depth analysis of whether the core innovation TA-OPD effectively mitigates task-related gradient conflicts when parameters are shared.
  3. The specific contribution of the OPD method itself has not been isolated.
Questions for authors
  1. Could you provide additional controlled experiments and ablation studies to address the concerns raised above?
  2. Could you provide a more detailed disclosure of the training data?
  3. Could you explain more about the figure 3?
Reviewer_3uyZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the rollout costs of high-resolution generation, this paper proposes a causal video model that learns world prediction and high-resolution refinement in shared parameters. Extensive experiments show that SCALEWORLD effectively unifies prediction and refinement while reducing the context-storage cost of high-resolution world generation.

Critical strengths
  1. The authors address an important efficiency problem in high-resolution generation tasks.
  2. The proposed self-cascading design makes sense to me.
Critical weaknesses
  1. The backbone is limited to a single 5B backbone, and the evaluation relies primarily on a single navigation-oriented benchmark with only 158 cases. I recommend the author verify the method's robustness across larger backbones and longer generation horizons.

  2. The paper requires first training a shared base model, followed by training two specialist models, and finally performing consolidation. The authors do not quantify the total GPU-hours, the number of training steps for each stage, or the relative training costs. And I suggest adding the more detailed costs analysis.

  3. Table 6 is the recipe comparison and does not demonstrate the effectiveness of each module. Please provide the more detailed ablation analysis.

Questions for authors

I have no further questions. Please focus on the issues mentioned in the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.