- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a decoupling strategy to model the world state and the final video rendering. Specifically, a coding agent creates a world engine from user-defined rules. And a video diffusion model renders final videos given the bounding box video priors. To achieve this goal, the paper proposes an interface named "Object-Centric State Interface" to bridge the box world state and the final video rendering. The paper also introduces a data and training paradigm to transform an autoregressive video generation model into a box-state-aware video generation model while trying to maintain long-term and real-time generation, which are important in current video world models.
Critical strengths
(a) The decoupling strategy, where a coding agent explicitly records rules and parameters, and a video diffusion model renders final videos given bounding box video priors, is essential for world simulation that requires precise control, long-term consistency, and so on. The paper leverages the advantages of both tools, i.e., the coding agent is easy to maintain precise rules and parameters, while the video diffusion model is powerful in rendering high-quality videos.
(b) While transforming an autoregressive video generator into a box-state-aware video renderer, the paper also manages to maintain real-time interaction, which is important to current video world models. Besides, the final rendered videos cover a diverse range of environments, verifying the generalization ability of the proposed method.
Critical weaknesses
(a) It seems that the quantitative results are not comprehensive enough. The paper compares its own model with LingBot-World-v1, LingBot-World-v2, and LongLive. And LongLive also serves as the base model of BoxWorld. I understand that there are many concurrent world model works trying to solve similar problems. And comparing to all of them seems to be impossible. However, I think comparing to additional one or two representative works is still necessary, such as HY-World 1.5, Matrix-Game-3.0, YUME 1.5, and Astra.
(b) Current video world models are also required to generate multi-turn and long-term video results. And some representative models can achieve mintue-level generation. However, the screenshot sequences shown in the paper and the videos shown in the supplementary materials seem to be relatively short.
Questions for authors
(a) See Critical Weaknesses (b). Current autoregressive generation often suffers from error accumulation, which leads to visual quality degradation. I am concerned whether the method can truly generate multi-turn long videos while maintaining visual quality and consistency.
Author Response
Thank you for your positive rating and recognizing that our decoupling framework is essential and effective! Below, we provide a point-by-point response to address your concerns. We welcome further discussion to enhance the clarity and effectiveness of our work.
A1:
A2(a): PairedStateBench evaluates multi-turn interaction: State-Fid checks the status after every turn, and Task requires all turns to succeed in order.
A2(b):