- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Different from many existing video world models that encode the states of identities and environments in implicit video latents, the paper presents a decoupling strategy to model several state attributes explicitly via coding agent programming, and produce final videos via coarse OBB video prior and video diffusion models. This strategy provides a more controllable, deterministic, and consistent way to maintain the world states and rules. This strategy also helps the video diffusion model to produce more structure-aligned videos, resulting in better long-term spatiotemporal consistency.
Critical strengths
(a) The proposed strategy, which models the state attributes explicitly via coding agent programming and renders final videos via coarse OBB video prior and video diffusion models, is essential for world simulation that requires precise rules, interactions, long-term consistency, and so on. Recent advances of the coding agent demonstrate the potential in image-conditioned 3D object/scene modeling with precise parameter control. The paper also uses this paradigm to strengthen the implicit video world model, leveraging the coding and the video diffusion model to handle the aspects where they have advantages, respectively, i.e., the coding agent is easy to record precise parameters and rules, while the video diffusion model is powerful in rendering high-quality videos.
(b) The paper provides a full-stack pipeline covering a dataset pipeline, a model and training pipeline, and a benchmark. Each part is important in generative model post-training paradigms. The model architecture is also reasonable. The chunk-wise autoregressive long video generation and 3D spatial memory mechanism are widely accepted in the video world model community.
Critical weaknesses
(a) It seems that the quantitative and qualitative results are not enough. And no ablation study is found. For quantitative results, I realize that there are a lot of concurrent works that are tackling similar questions. And comparing to all of them is impossible. However, comparing with some representative models is still necessary, such as HY-World 1.5, Matrix-Game-3.0, LongVie 2, and Astra. For qualitative results, the screenshots in the paper and the videos in the supplementary materials seem relatively simple. For example, the screenshots mainly show a third-person perspective to execute CS-style gunshot behaviors in diverse environments. I realize that the screenshots may come from the visualization of benchmark cases for convenience. However, I think the visualization of more diverse scenario settings is helpful to state "World Model".
(b) The paper is titled "Programmable World Model". However, it seems a bit of over statement, since the demonstrated results are mainly third-person gunshot game videos, and the state attributes are relatively limited. The ability to model a more realistic world is somewhat questionable. I think reducing the scope of the statement would be helpful to clarify the content of the paper. For example, "Programmable Game World Model" is more precise. And I think the scope of "Game World Model" is already enough to be investigated solely in a research paper.
Questions for authors
(a) Real-time interaction is essential in world model concepts. Although early works did not emphasize this point, many later video world models have achieved real-time interaction. The paper uses the chunk-wise autoregressive method to achieve multi-turn long video generation, where the bi-directional denoising is executed in a chunk, and causal denoising is performed across chunks. This paradigm itself is widely accepted in the video world model community. However, it seems that the paper does not discuss acceleration on the proposed model, making the gap between the current inference latency and the real-time interaction unclear. And it is also unclear how quickly the model can respond to input interactions. How long is the temporal latency between user action input and rendered video output?
(b) Long-term generation and consistency are also important in world model concepts. However, current autoregressive generation often suffers from error accumulation, which leads to visual quality degradation. The maximum length of the video results in the supplementary material is about 30 seconds, which is relatively too short compared to many existing minute-level video world models. Besides, while the coding agent can maintain world rules and parameters, it is unclear whether the video diffusion model can render long videos with acceptable error accumulation. I am concerned whether the decoupling strategy can truly generate multi-turn long videos while maintaining consistency.
5 comments
Response to Reviewer BRhZ (2/3)
W(a) — Added HY-World 1.5, Matrix-Game 3.0, and Astra as baselines, a racing benchmark, new interaction types and viewpoints, and component ablations.
Additional baselines. We added HY-World 1.5, Matrix-Game 3.0, and Astra to the comparison, alongside LingBot-World v2 and YUME. Table 1 and Figure 3 compare all five baselines with our method on CombatStateBench. Our method ranks first on all eight state-consistency and entity-level control metrics. We did not include LongVie 2 because it addresses a different input setting: it is conditioned on dense depth maps and sparse point maps of the target sequence, which presuppose the geometry of the frames to be generated. Our setting provides only an initial image and programmed OBB trajectories, from which such controls cannot be derived. We discuss LongVie 2 in Related Work.
A racing benchmark. We also compared all methods on a racing benchmark with 27 clips and 11 programmed maneuvers, including braking, spinning, cutting in, and overtaking (Appendix E.1, Table 2, Figure 6). Our method again ranks first on all eight state-consistency and entity-level control metrics. Unlike combat, where states change discretely from alive to dead, racing requires following continuous programmed trajectories and vehicle-to-vehicle interactions. The advantage of our method therefore extends beyond combat-specific state transitions.
Additional scenarios and viewpoints. Following the reviewer's suggestion to broaden scenario and viewpoint coverage, Figure 7 and Appendix E.2 present sitting and standing, object transfer, entering and driving a vehicle, pushing and releasing an object, and first-person traversal. These examples demonstrate the same Programmable World Model framework instantiated with MiniMax-H3 as the generative renderer. A coding agent generates executable world rules, and the engine evolves state-augmented 3D OBBs. The compiler converts these states into a camera-aligned box-control video, supplied together with the initial RGB image and timestamped action descriptions through MiniMax-H3's reference-conditioning interface. These examples provide qualitative evidence of the framework's applicability to additional interactions and viewpoints.
Component ablations. We added two sets of studies in Supplementary Appendix G (Tables 3–4 and Figures 10–11).
Temporal history and spatial memory (Table 3 and Figure 10). These components aim to improve temporal consistency. We therefore measure frame-to-frame visual stability with Temporal stability, and preservation of entity identity at programmed locations over time with Identity at GT location. We evaluate three configurations on ten 30 s clips, using the same initial images, prompts, and programmed controls.
| Configuration | Temporal stability ↑ | Identity at GT location (%) ↑ |
|---|---|---|
| Without history (last-frame chaining) | 98.85 | 61.8 |
| With temporal history | 99.02 | 68.1 |
| With temporal history + spatial memory | 99.15 | 78.0 |
Each component improves both metrics, and combining them performs best. The effect is clearest on Identity at GT location: temporal history raises it from 61.8% to 68.1% (+6.3 points), and spatial memory raises it further to 78.0% (+9.9 points). Temporal stability is already high without history (98.85) and increases steadily to 99.15. Both components therefore mainly help preserve entity identity over long rollouts, with spatial memory contributing the larger gain.
OBB conditioning and state augmentation (Table 4 and Figure 11). This ablation evaluates OBB conditioning and its state augmentation. We choose Entity box IoU to assess spatial alignment, Motion direction to assess adherence to programmed movement, and Re-entry similarity to assess appearance consistency across exit and re-entry. The three variants use the same training duration and the same 14 evaluation scenarios, with shared initial images, prompts, and camera trajectories.
| Configuration | Entity box IoU ↑ | Motion direction (%) ↑ | Re-entry similarity ↑ |
|---|---|---|---|
| Camera only | 21.5 | 11.9 | -- |
| 3D OBBs without state augmentation | 57.4 | 83.7 | 0.644 |
| Ours: state-augmented 3D OBBs | 62.7 | 85.2 | 0.751 |
Our full state-augmented OBB configuration performs best on all three metrics. Re-entry similarity is the cosine similarity between a vehicle's mean DINOv2 appearance features before exit and after re-entry, averaged over eligible events. The higher re-entry similarity (0.751 vs. 0.644) provides evidence that state-augmented 3D OBBs better preserve global identity information across exit and re-entry than geometry-only 3D OBBs, helping entities retain a consistent appearance after being out of view. Detailed metric definitions and evaluation protocols are provided in Section 4.2 and Supplementary Appendices D and G of the revised paper.
Response to Reviewer BRhZ (3/3)
W(b) — Clarified the scope: validated mainly in game environments, with broader settings left to future work.
We agree with the reviewer that our validation is conducted mainly in game environments and that the current results do not establish the ability to model more realistic worlds. We will narrow the scope of our claims accordingly.
Within games, the revision broadens the demonstrated settings beyond third-person shooting. The racing benchmark (Table 2 and Figure 6) evaluates continuous vehicle trajectories and vehicle-to-vehicle interactions, and Figure 7 adds posture, object possession, vehicle occupancy, and object contact, as well as a first-person viewpoint.
However, the framework itself is not specific to games. It separates an explicit, executable world state from its generative visual realization, and its state interface supports persistent entities, attributes, relations, and executable rules (Section 3.2 and Figure 2). This design may extend to other domains with structured entities and rules, but we have not validated such settings and leave them to future work.
Q(a) — Action-to-output latency: about 122 ms on one H200.
We verified that the proposed framework can support real-time interaction. To this end, we reduced the chunk size to one latent frame, corresponding to four video frames, distilled a two-step student, and applied engineering optimizations, including CUDA Graphs, fused kernels, FP8 attention and linear layers, a ring-buffer KV cache, and a lightweight VAE. With model inference running on a single H200 GPU at 832 × 480 and a 60 FPS input stream, the system achieves a mean action-to-output latency of about 122 ms.
Of this total, about 50 ms is inherent to processing four-frame chunks at 60 FPS, including waiting for the remaining input frames and presenting the four output frames in order. Model inference accounts for about 36 ms, covering encoding, two denoising steps, and decoding. The remaining 36 ms comes from other overhead, including waiting for the next simulation step, rendering the conditioning frames, and encoding, decoding, and transferring frames.
Q(b) — Added a 60 s rollout with no visible style drift.
We added a 60 s autoregressive rollout, using the same model and generation procedure as the original 30 s tests. Supplementary Appendix F.2 and Figure 9 present generated frames alongside the 3D OBB controls at 1, 15, 30, 45, and 60 s, with forward camera motion and rotation. In this 60 s example, we observe no obvious stylistic drift. This single case does not quantify error accumulation or establish robustness across longer rollouts.
Response to Reviewer BRhZ (1/3)
We thank the reviewer for the suggestions on baseline coverage, scenario diversity, scope, interaction latency, and longer-horizon rendering. We address these points separately below. All additions made to the revised manuscript during the rebuttal are marked in blue.