acceptodds
Withdrawn from ICLR 2027

A Discrete World Exploration Model with Superior Action Alignment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As a fundamental world-modeling task, world exploration requires accurate action alignment and stable long-horizon generation. Existing approaches are predominantly diffusion-based but still exhibit limitations, particularly in accurate action alignment. Moving beyond diffusion, we introduce DiscreteWorld, a discrete generative model that produces stable 10-second, 480p, camera-controllable open-world videos with competitive action controllability. A key design of DiscreteWorld is to quantize both camera actions and visual observations into a unified discrete token space for joint reasoning and generation, yielding two essential advantages. First, unified action-vision modeling in the same token space enables precise semantic alignment between the two modalities, leading to accurate and reliable action following. Second, viewing discretization as a form of refinement and compression, we conduct the entire world modeling—including semantic understanding and video generation—in a discrete token space to facilitate the learning of higher‑quality latent compressed representations, which is crucial for world models. Additionally, we build our DiscreteWorld upon GRNs—a text‑to‑video discrete generative model—and thereby naturally inherit its iterative visual refinement capability for high‑quality video synthesis. Extensive experiments on real-world and game scenarios show that DiscreteWorld achieves more accurate action following and more stable long-horizon generation than prior diffusion-based world models. All models and code will be open-sourced to foster further research.

Primary area:
generative models
Keywords:
World Model, Generative Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QQErOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DiscreteWorld is to quantize both camera actions and visual observations into a unified discrete token space for joint reasoning and generation, yielding two essential advantages. First, unified action-vision modeling in the same token space enables precise semantic alignment between the two modalities, leading to accurate and reliable action following. Second, viewing discretization as a form of refinement and compression, the authors conduct the entire world modeling—including semantic understanding and video generation—in a discrete token space to facilitate the learning of higher‑quality latent compressed representations, which is crucial for world models. Additionally, the authors build our DiscreteWorld upon GRNs—a text‑to‑video discrete generative model—and thereby naturally inherit its iterative visual refinement capability for high‑quality video synthesis. Extensive experiments on real-world and game scenarios show that DiscreteWorld achieves more accurate action following and more stable long-horizon generation than prior diffusion-based world models.

Critical strengths

The paper addresses action controllability and long-horizon consistency, two important challenges in world exploration. The motivation for exploring discrete generative modeling beyond conventional diffusion-based approaches is clear and relevant. And the overall framework is well organized, the main components are relatively easy to follow.

Critical weaknesses
  1. The proposed method largely inherits HBQ, iterative discrete refinement, and visual token generation from GRNs. Its main contributions appear to be discretizing camera actions and introducing an anchor-history-prediction structure. Therefore, the proposed unified action-vision modeling resembles an action-conditioned extension of GRNs rather than a fundamentally new world-modeling mechanism. The authors should better clarify the methodological novelty beyond the existing GRN framework.

  2. The paper lacks direct evidence explaining why discrete modeling improves action-vision alignment. Improvements in downstream performance do not necessarily establish the underlying mechanism, and representing both actions and visual observations as discrete tokens does not inherently guarantee stronger semantic alignment between them.

  3. Although the paper emphasizes stable long-horizon generation, the main comparisons are limited to 10-second videos. It remains unclear whether the model can preserve scene consistency over longer trajectories, particularly when revisiting previously observed locations. For example, can the model accurately reconstruct an object encountered at 5 seconds when the camera returns to the same location at 20 seconds?

  4. The paper does not clearly explain how the different dimensions of the camera-action descriptor are normalized before quantization. Moreover, it remains unclear whether action and visual features share the same HBQ quantization parameters or use separate quantization mechanisms.

Questions for authors

Please refer to the weakness.

  1. Please explain the novelty beyond the existing GRN framework.
  2. Please give some direct evidence explaining why discrete modeling improves action-vision alignment.
  3. Whether the model can preserve scene consistency over longer trajectories, particularly when revisiting previously observed locations. For example, can the model accurately reconstruct an object encountered at 5 seconds when the camera returns to the same location at 20 seconds?
  4. Please explain more details like how the different dimensions of the camera-action descriptor are normalized before quantization and whether action and visual features share the same HBQ quantization parameters or use separate quantization mechanisms....

If the authors adequately address the questions above, I would be willing to raise my score.

Reviewer_2pYFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors proposes a discrete generative model to produce stable and camera-controllable open-world videos. The main idea of the proposed pipeline is to quantise both camera actions and visual observations into a unified discrete token space for joint reasoning and generation. This unified approach enables precise semantic alignment between control signals and visual dynamics. Built on Generative Refinement Networks (GRNs), the framework treats discretisation as a form of compression to maintain high-quality latent representations for long-horizon prediction. Experiments demonstrates that DiscreteWorld achieves superior action alignment and greater long-horizon temporal stability across both real-world and game scenarios.

Critical strengths
  • The main idea (quantising both camera actions and visual observations into a single discrete token space) improves semantic alignment between control signals and visual dynamics

  • The proposed method shoes low trajectory errors compared with other methods.

Critical weaknesses
  • Integrating camerawork and visual observation into a unified token space is a good attempt, but its practical advantage over separated camerawork is unclear. A demonstration video linking user input directly to the output is needed. In Figures 1, 3, and 4, the results from comparative methods do not fail at video generation but just output incorrect camerawork. For example, in Figure 4, GT and "Ours" show a tracking shot (dolly shot), while the camera moves forward faster than the subject in the other SOTA methods. However, since a simple "W" key input does not clarify the user's intended camerawork, it is hard to objectively judge "Ours" is better than others (There is no point of GT in this case if the function of “W” is not specifically defined and delivered to the system in advance). The same applies to Figure 1. Yume's result simply looks like the key input was incorrectly passed to the generation process. In actual production, a director's intent MUST be specified much more precisely. Since explicitly defining camerawork is straightforward, I don’t think this setup serves a fair comparative evaluation.

  • Figure 2 is difficult to understand on its own. It is hard to trace the input, connections, and output flow, and the main text does not provide detailed explanation of this structure.

  • Figure 2 and Section 4.2 indicate that the history chunk consists of 1 sec future frames rather than past frames from the anchor. This makes the system non-causal. Non-causal systems have many limitations compared to causal systems and are fundamentally unsuitable for real-time applications, even if speed issues are eventually resolved. The rationale for using future 1 sec frames instead of past frames as the history chunk should be provided.

  • Without a demonstration video, the smoothness of the generated videos cannot be evaluated only from the provided snapshots.

  • The subjective evaluations lack statistical validation (e.g., P-values, ANOVA).

Questions for authors

Need clear explanation or justification for the points mentioned in Weakness

Reviewer_k3TROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DiscreteWorld that adapts a generative refinement network to camera-controlled video prediction. Camera actions and visual observations are quantized into binary tokens and processed jointly. Generation proceeds in overlapping chunks, conditioned on a persistent first-frame anchor and recent history. The paper compares continuous and discrete formulations under matched settings, evaluates the resulting system on real-world and game videos, and examines longer rollouts and individual design choices.

Critical strengths

Table 1 is the most convincing part of the paper. The authors do not rely solely on comparisons between differently pretrained systems: they also train variants with the same Transformer architecture and optimization budget. Within GRN, discretizing actions reduces rotation error from 3.35 to 2.81 and translation error from 7.87 to 6.98. This provides evidence for the action representation beyond simply replacing a diffusion backbone with GRN.

The end-to-end results are encouraging as well. DiscreteWorld has the lowest reported trajectory errors in both scene domains. I appreciated that the paper distinguishes this advantage from visual quality, where it does not win uniformly. The 30-second evaluation is also informative: action errors remain stable while appearance quality deteriorates. That is a useful account of what the system can and cannot maintain during rollout.

Critical weaknesses

Some explaintations do not have enough evidence ro support. For exaple, Table 1 supports discrete action conditioning within GRN, but the cross-generator comparison changes visual discretization and the training objective together. Also, sharing a binary representation does not, by itself, establish semantic understanding. I would frame the contribution around the demonstrated action-following improvement and distinguish that result from the proposed explanation for it.

The human evaluation seems less persuasive. Firstly, the paper specifies 100 participants and 50 held-out videos per domain, but not how judgments are assigned to clips or aggregated. Secondly, the real-world human-score difference from Yume is 72.3% versus 67.8%, so uncertainty matters here. Table 7 also lists different control interfaces without explaining how equivalent instructions were constructed. These omissions make the human comparison harder to interpret than the controlled experiment.

Questions for authors
  1. In Table 1, do continuous and discrete action inputs use the same token count, embedding width, and conditioning pathway? Please also specify the normalization applied before action quantization.
  2. How were equivalent controls presented across the different interfaces, how many judgments did each clip receive, and were model identities hidden? Can you report uncertainty for the paired human-score differences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.