- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents ReactiveGWM, a video world model for controlling multiple characters and generating NPC reactions in games. At the start of each rollout, users choose which characters to control directly and which should follow given NPC rules. Spatial Role Binding links characters in the initial frame to their control inputs, while Unified Agency Conditioning handles both player action sequences and conditional NPC rules. The model is fine-tuned from a pretrained video generator and evaluated in two shared-view 2D games. The experiments assess action execution, character selection, and NPC reactions, showing separate control of characters with similar appearances and NPC responses triggered by player actions. Ablations also show the contribution of role binding and NPC rule conditioning.
Critical strengths
The paper explores world models with multiple interacting NPCs, which I think is a promising direction. Judging from the examples, these are among the better demonstrations of multi-NPC interaction I have seen. They show NPC reactions triggered by player actions, as well as multiple characters performing different actions in the same scene.
There are also some interesting design choices. The authors use the same character handle to connect a character’s visual region with its control inputs, and handle player actions and NPC rules through the same control mechanism. This allows users to assign direct control or rule-based behavior even to characters that look identical. The design is straightforward, and the ablations show that it helps.
Critical weaknesses
I work on both coding agents and video generation, and my impression is that coding agents are progressing faster at the moment. This leaves me with some questions about the practical value of this approach. I am not yet sure which would be cheaper and more stable: using a coding agent through its API to build a similar game from scratch, or building a video generation framework, collecting data, and training a world model. The paper evaluates temporal consistency, but the experiments are limited to five-second clips. It also does not report training and inference costs, interaction latency, or behavior over longer runs. I find the work interesting from a research perspective, but for actual game development, the additional benefits over implementing similar interactions in code still need to be explained.
Questions for authors
Please refer to the Weaknesses section.
5 comments
Thank you for providing the longer demonstrations, training costs, and inference speeds. These additions address some of my concerns about missing experimental details, but they do not resolve my central concern about the practical value of this approach. I still find coding-agent-based game development more promising in terms of stability, extensibility, and sustained gameplay. I would appreciate clarification on four concrete scenarios:
-
During prolonged simultaneous two-player control, especially when characters overlap or exchange positions, do identity swaps or control errors accumulate over time?
-
Can NPCs combine multiple rules into more complex gameplay strategies or follow condition–response combinations not represented in training?
-
Can users control previously unseen characters using only a new initial image and character masks, or is additional training required?
-
How does the model retain persistent changes caused by earlier interactions, such as damage to characters or changes to scene objects, once the relevant events fall outside its current video context?
My broader concern is that the paper primarily demonstrates training a video model on data collected from specific 2D games, while providing limited evidence that the model reliably captures interaction rules and maintains persistent game state. Once NPCs and multiple players are introduced, consistent interactions and reliable rule execution become central to the gameplay experience. The additional 30-second demonstrations are helpful, but extending a demonstration does not establish reliability during sustained play. The current results strike me as a promising research demonstration, rather than sufficient evidence for the more flexible game-development possibilities described in the response.
I also remain unconvinced that the authors have adequately justified using a heavyweight video-generation backbone for these settings. Code-based games can explicitly maintain state and rules while supporting targeted debugging and incremental feature development. The video-generation approach additionally has to address visual degradation, long-term memory, and consistency over extended rollouts. Although the response highlights the potential advantages of fine-grained motion and rich visuals, it does not establish what concrete experiences these advantages enable in the demonstrated 2D settings that would be difficult to achieve through code.
Cost further strengthens this concern. For the relatively limited 2D scenarios demonstrated here, using a coding agent to develop a playable game with comparable interactions may not require substantial development effort, and the resulting code can be reused and continuously modified. By contrast, the video-generation approach requires upfront data collection and training, followed by ongoing generation costs every time the game is played. Code-based games also incur execution and rendering costs, but the response does not establish that the additional generation costs deliver corresponding improvements in interaction capabilities or gameplay experience.
The response therefore has not persuaded me to change my overall assessment: the current results appear to reproduce interactions that code can already implement reliably, using a complex and computationally expensive generative framework, without demonstrating sufficient additional value to justify that choice. I remain pessimistic about the practical prospects of this approach to 2D game development. I maintain my rating of 3: Weak acceptance in recognition of the paper’s specific research contribution to multi-character control and conditional NPC reactivity, rather than agreement with the application prospects presented in the authors’ response.
Thank you for your timely reply and for recognizing the scientific value of our work. We first address your four new questions:
- We added an 30-second rollout with separate control streams for both characters, shown in
Anonymous Project Page. It shows an initial position exchange and further interaction, but also ghosting, unintended jumps by the non-target character, and failed later crossings. These failure cases indicate that control errors indeed accumulate during prolonged interaction. - We tested the distilled model on combined rules without additional training. The results suggest that reliable multi-rule generalization has not yet been established. This limitation is understandable, as training relies on single-rule rollouts with only ten predefined NPC rules per game, offering limited generalization to unseen condition–response combinations. We emphasize that our work primarily focuses on flexible character control and local NPC reactivity, while more complex NPC planning and compositional rule generalization remain outside the scope of this work.
- We tested the original ReactiveGWM with a new initial frame replacing both characters and the scene, together with corresponding masks, as shown in
Anonymous Project Page. The background and the new character portraits in the upper-left and upper-right corners remain largely stable. However, the characters become blurred or distorted when performing actions. - Our model does not include an explicit game-state or event-memory module. Our focus is flexible character control and local NPC reactivity and long-term state and event consistency remains a limitation outside the scope of this work.
Regarding application value, we agree with your point: code agents can implement these 2D games with explicit mechanics, NPC logic, and persistent state. Our paper focuses on how a learned video world model can bind external actions and conditional NPC behaviors to individually specified characters. We use 2D games primarily because their clearly defined actions, character identities, and interaction rules provide a controlled setting for evaluating this research question. We hope this character-level interaction interface can support future interactive video generation and visual prototyping. We do not claim a demonstrated development-cost or reliability advantage over code-based implementations.
Thank you again for your thoughtful feedback. If you have any further questions or concerns, please let us know.
Author Response
>>>Q1Research Contributions vs. Practical Value of coding agents Approaches>>>A1Thank you for the insightful question! We agree that coding agents can implement predefined game mechanics more efficiently and reliably. However, video world models offer the potential to enable more flexible and fine-grained interactive experiences, expanding the possibilities of game creation beyond fixed, explicitly programmed mechanics. For example, while code-based engines can efficiently define game logic and coarse scene structures, video world models can generate fine-grained character motions and detailed visuals.Moreover, recent research has begun exploring hybrid approaches that combine code-based engines with video world models, aiming to leverage the complementary strengths of both paradigms (e.g., Code World Model and Matrix-Game 4.0).
Our work focuses specifically on advancing video world models. We take a step toward this vision by exploring flexible multi-character control within a unified video world model, enabling characters to be either controlled by players or exhibit conditional NPC reactivity. We hope our research contributes to the development of video world models toward richer, more flexible, and interactive virtual worlds.
>>>Q2Longer video and Training/inference cost>>>A2Thank you for the suggestion! For longer video, we further train a streaming variant using the Causal Forcing++ algorithm.Appendix Fpresents two continuous 30-second demonstrations at approximately 10 FPS: (1) two externally controlled players executing multiple actions; (2) NPC repeatedly approaching the player after each retreat. Both demonstrations illustrate sustained character control and repeated interactions well beyond the five-second clips. The videos are shown in theAnonymous Project Page.For training and inference costs, training for 20K updates takes approximately 160 H200 GPU-hours, while the separate streaming adaptation requires approximately 300 H200 GPU-hours. For inference, the original model runs at approximately 2 FPS, whereas the distilled streaming model achieves approximately 10 FPS, corresponding to average per-frame processing times of approximately 500 ms and 100 ms, respectively, both including VAE decoding.