acceptodds
Desk rejected at ICLR 2027

Streaming-AAA: Streaming AAA Game Video Generation at High Resolution, High Frame Rate, and Low Latency

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Controllable streaming video generation offers a promising route toward interactive game simulation, yet existing methods remain far from practical gameplay. They often fail to jointly achieve high visual fidelity, high frame rates, and low interaction latency, resulting in poor responsiveness and degraded user experience. We present Streaming-AAA, a new paradigm for controllable infinite generation of AAA-quality game videos. Our system generates 1080p gameplay videos at 43 FPS with 198 ms end-to-end latency interaction on merely NVIDIA A800 GPUs, while supporting long-horizon play without severe temporal degradation or collapse. Streaming-AAA is built upon three key designs. First, we redesign the VAE architecture for efficient high-resolution decoding, enabling real-time synthesis at 1080p. Second, we pretrain the diffusion model using diffusion forcing and significantly extend the temporal training horizon, allowing the model to capture long-term dependencies and maintain persistent memory. Third, we perform post-training distillation and memory compression to improve inference efficiency while retaining visual quality and temporal consistency. Experiments show that Streaming-AAA substantially improves visual fidelity, frame rate, latency, and long-term stability over prior controllable video generation approaches, enabling a significantly more playable experience for interactive AAA game video generation.

Primary area:
generative models
Keywords:
Video Generation, Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uoYJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Streaming-AAA generates interactive streaming video of AAA game environments using a high-resolution video VAE and an SD-1.5-initialized U-Net with temporal modules. It combines long-context Diffusion Forcing pre-training with DMD post-training, using a bounded memory of recent and distant frames during generation. By applying inference optimizations and pipelining the diffusion model and VAE decoder across separate A800 GPUs, the system achieves 43 FPS at 1080p with an estimated 198 ms action-to-pixel latency. Evaluated on Genshin Impact and Honor of Kings: World, it outperforms baselines on the mean of four selected custom-input VBench metrics and maintains visual stability across minute-long comparisons and one-hour demos.

Critical strengths

The system combines high-resolution output with interactive streaming speed. Meeting these constraints together is a useful engineering contribution for game video generation. Moreover, the minute-long comparisons and one-hour demonstrations show sustained visual stability in the evaluated games. Several baselines degrade much sooner in the comparisons, so the results support the use of this system for longer interactive sessions.

Critical weaknesses
  1. The paper's core contribution is system integration rather than algorithmic novelty. The video VAE, factorized spatiotemporal U-Net, long-horizon training, and hybrid memory all follow standard designs. While adapting the DMD perturbation scheme to a DF-pretrained model is a potential technical contribution, but it lacks direct validation. Existing ablations evaluate post-training and horizon length overall, but never isolate the impact of independent per-frame noise levels against a shared noise level.

  2. Generalization is insufficiently evaluated, both across games and to unseen regions within the training game. First, retraining the pipeline on Honor of Kings: World only demonstrates recipe reusability rather than cross-game transfer of a trained model, while brief out-of-domain continuations do not establish action responsiveness or long-horizon stability. More importantly, the paper fails to clarify whether test scenes are separated from training data within each game, leaving it unclear if performance reflects genuine generalization to unseen map regions or mere memorization of recorded training areas.

Questions for authors
  1. Can you provide an ablation comparing independent per-frame noise levels against a shared noise level during DMD (with rollout, training horizon, and memory policy fixed) to isolate the benefit of this noise sampling scheme?

  2. How does a trained model perform zero-shot on unseen games without retraining or fine-tuning? Please provide continuous, action-controlled rollouts or quantitative metrics to clarify out-of-domain visual quality, control responsiveness, and stability.

  3. Are the test scenes strictly held out from the training map regions? Please clarify the train/test spatial split and report performance specifically in unseen map areas.

Reviewer_MUmvOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an action-conditioned, UNet based, Causal (Streaming) Video generation pipeline titled StreamingAAA. The reason why I do not call it a World-Model, is because this one is a per domain fitting model as opposed to the more general text/image conditioned video world models like the HY-World, etc. Specifically the authors train two flavors on somewhat similar looking video games, Genshin Impact (main model), and Honor of Kings: World (supplementary), which makes it quite unclear as to what purpose or towards what deployment setting such a model could be useful.

Critical strengths

The paper is well written, and much easier to follow compared to the other papers I have reviewed this time. The flow is nice, and sections are meaningfully divided into informative chunks, allowing the reader to focus on one part of the pipeline at a time.

Critical weaknesses
  1. I believe the NeRF-like per game fitting paradigm of this proposed method is the biggest limitation. Mainly because the model is only learning the world state, rendering system, and rudimentary physics of the underlying game, and is not at all learning the game-mechanics, level-design, and other sophisticated game-systems that are a part of typical AAA video games, RPG or otherwise. A singular video-game styled world modeling doesn't offer specific advantage over just being able to walk around and jump across 3D worlds for hours. The paper does a good job explaining well what they have actually done, but I believe it fails to motivate the actual scenario/use-case that they are after.

  2. This point then trickles down into a major scientific issue, which is comparing to the 4 true World-Model baselines is not fair on various axes. a. First, all other comparisons are to true-world models, for instance Yume1.5 [1] allows for both text-and-first frame generalizable control. So do the rest of the baselines [2, 3, 4] either through text or image. While StreamingAAA is only trained on Genshin Impact, and evaluated on Genshin Impact, as opposed to others which are trained on much more diverse settings and scenarios. b. Second, all others are Transformer based backbones (DiTs) which StreamingAAA is UNet based, so the latency being better is not really surprising. The ideal comparison should be: at-least on one of the baselines to match the backbone UNet, and trained equally on you Genshin Impact dataset. c. Whereas some of the baselines do provide options of world interactivity, StreamingAAA is limited to subject movement, and world look-and-feel controllability via weather and time-of-the-day conditioning.

  3. More on the demonstrated results, there are still several shortcomings: a. In spite of being such a specialized training setup, as a single video game, there are significant visual artifacts that renders deployment for a real-world use-case impractical. Not to mention the requirement of A100 gpus, which are far far far away from consumer grade hardware. b. The authors evaluate their results with VBench, which should be done probably, but because you do have a deterministic game setup here, you could actually evaluate true signal processing first-class metrics like PSNR/SSIM in this case. I believe these would still be much lower like in the range of 15-20, and that will clearly demonstrate how far this model is from actual interactive use-cases. Also, I feel like the chosen dimensions from VBench are also a bit mis-representative. For instance subject consistency will be high by design, because all frames will always have the video-game character subject. The background consistency for instance is the more telling axis in this case. But nevertheless, neither of the results shine light on anything new.

  4. Finally, the proposed method has no new ideas and/or insights. a. The highly compressive VAE is not new. Many papers [5, 6, 7, 8, 9, 10] have at this point proposed ways of using high spatial compression, but neither of them have been able to temporally compress the latents significantly. This could have been a good opportunity for StreamingAAA to make actual impact here (something similar to Nvidia's FrameGen), but they miss out on it, by sticking to simple 4x temporal compression. b. The Diffusion Forcing based pretraining and DMD post-training is an interesting design choice, but I believe has been wasted on UNet. Except for highly specialized and commoditized use-cases, there is no demand for UNets in the industry at the moment. More and more OEMs, as well as deployment settings are asking for efficient DiTs over UNets. Again, the long-horizon training is actually possible, because of the tiny <2B UNet that the authors are training, and not because of any special cool-new tricks from authors that enables this.

References: [1] Mao, X., Li, Z., Li, C., Xu, X., Ying, K., & Zhang, K. (2026). Yume1. 5: A text-controlled interactive world generation model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7752-7761). [2] Sun, W., Zhang, H., Wang, H., Wu, J., Wang, Z., Wang, Z., ... & Guo, C. (2025). Worldplay: Towards long-term geometric consistency for real-time interactive world modeling. arXiv preprint arXiv:2512.14614. [3] Wang, Z., Liu, Z., Li, J., Huang, K., Xu, B., Kang, F., ... & Zhou, Y. (2026). Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory. arXiv preprint arXiv:2604.08995. [4] Team, R., Gao, Z., Wang, Q., Zeng, Y., Zhu, J., Cheng, K. L., ... & Ouyang, H. (2026). Advancing open-source world models. arXiv preprint arXiv:2601.20540. [5] Wu, Y., Zhang, Z., Li, Y., Xu, Y., Kag, A., Sui, Y., ... & Ren, J. (2025, June). Snapgen-v: Generating a five-second video within five seconds on a mobile device. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2479-2490). IEEE. [6] Wu, Y., Li, Y., Skorokhodov, I., Kag, A., Menapace, W., Girish, S., ... & Tulyakov, S. (2025). H3ae: High compression, high speed, and high quality autoencoder for video diffusion models. arXiv preprint arXiv:2504.10567. [7] Wu, Y., Li, Y., Kag, A., Skorokhodov, I., Menapace, W., Ma, K., ... & Tulyakov, S. (2025). Taming diffusion transformer for efficient mobile video generation in seconds. arXiv preprint arXiv:2507.13343. [8] Karnewar, A., Korzhenkov, D., Lelekas, I., Fathima, N., Karjauv, A., Ghafoorian, M., & Habibian, A. (2026, April). Neodragon: Mobile video generation using diffusion transformer. In International Conference on Learning Representations (Vol. 2026, pp. 39506-39537). [9] Zhao, L., Wu, Y., Lebedev, A., Lahiri, D., Dong, M., Sahni, A., ... & Li, Y. (2026). S2dit: Sandwich diffusion transformer for mobile streaming video generation. arXiv preprint arXiv:2601.12719. [10] Ghafoorian, M., Korzhenkov, D., Karjauv, A., Lelekas, I., Fathima, N., Stasis, S., ... & Habibian, A. (2026). MobileWan: Closing the Quality Gap for Mobile Video Diffusion. arXiv preprint arXiv:2607.06173.

Justification: This paper contributes nothing except for providing a singular datapoint of somewhat working empirical evidence for the hypothesis that "A specialized UNet with specialized training for a singular specialized task (Genshin impact game frame generation) could be better than the Transformer". Due to its limited impact, I therefore propose a weak rejection.

Questions for authors
  1. Similar to the VAE, is the Diffusion UNet also trained from scratch? The paper mentions architectural cloning from SD1.5, but doesn't specify if started from those weights or not.

  2. If yes, then how do you ensure that the VAE latents end-up being close to the SD1.5 latents?

  3. What happens to SD1.5’s original text-prompt conditioning? Are the CLIP text encoder and U-Net cross-attention layers retained, removed, or repurposed for action conditioning?

Reviewer_8gnvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Streaming-AAA, a system for action-conditioned game video generation, with results on Genshin Impact and Honor of Kings: World. The long-rollout examples are promising, and the paper addresses an important challenge in interactive video generation. I have three main concerns about the evidence supporting its conclusions.

Critical strengths

The system and visual results are promising, but the current comparisons and evaluations do not yet establish the claimed advantages in controllability and long-term stability.

Critical weaknesses
  1. Baseline comparability is unclear. The authors collect data and train models for both target games (Section 4.1 and Appendices A/C), but state that the baselines use their original code and released checkpoints (Section 4.1). If the baselines did not receive comparable training or adaptation on these games, the quality gaps in Tables 1 and 2 cannot be attributed solely to the proposed method. The paper also does not explain how each baseline receives the same keyboard, mouse, weather, and time-of-day conditions, or whether latency is measured under comparable hardware and timing protocols. Please provide per-baseline details and a comparison with more closely matched training conditions.
  2. Evidence for playability and “infinite” stability is limited. The VBench metrics in Tables 1 and 2 primarily assess visual properties; they do not measure whether the generated video follows movement, jumping, or camera controls correctly. Figures 8 and 14 show one frame every five minutes from the hour-long rollouts, which cannot reveal temporary collapse, loss of control, or scene drift between samples. Please report action-following accuracy, measured interaction latency, and quality over time, and provide continuous rollout videos. An apparently stable one-hour rollout does not establish indefinite stability.
  3. The ablations do not isolate the key contributions. Figures 6 and 15 mainly compare pre-training, shorter post-training, and the full model. The full model changes training length and memory configuration together, making their individual effects difficult to assess. The separate contributions of per-token noise perturbation, hybrid memory, extended training, and the VAE design also remain unclear. Controlled ablations reporting both quality and speed would make the technical claims substantially stronger.
Questions for authors
  1. Were the baselines fine-tuned on either target game? How was the same action sequence mapped to each method’s control interface?
  2. Can the authors separately ablate training duration, hybrid memory, per-token noise perturbation, and the VAE design, reporting both quality and speed?
  3. Can the authors report action-following accuracy and time-resolved quality measurements, and provide continuous videos of the hour-long rollouts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.