- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MultiGen decomposes a diffusion-based game engine into Memory, Observation, and Dynamics modules. Memory stores 2D wall geometry and player poses; Observation generates the next frame from ray-cast disparity, recent images, and actions; Dynamics predicts pose increments from diffusion features and state information. The multiplayer version shares player poses and alive/dead states across model instances, with opponent-aware geometry and a kill-prediction head. Experiments in Doom/ViZDoom evaluate layout control and multiplayer visual consistency through image metrics, user preferences, opponent visibility, and bounding-box IoU. The reported real-time rate is approximately 20 FPS with one A100 per player, rather than a constant-hardware guarantee as the player count increases.
Critical strengths
A sensible decomposition with a useful control interface. The static map, evolving poses, and image generator have clear responsibilities. Direct map editing and shared state access place both applications within the same architecture. The paper offers a concrete interaction interface, and this system contribution has practical value beyond simply producing longer videos.
Critical weaknesses
The reliability claims exceed the scope of the quantitative evaluation. The level data comprise 100 maps, but map-disjoint training and test sets are not clearly specified, leaving unseen-layout generalization unresolved; multiplayer training and evaluation explicitly use one map. The main quantitative rollouts extend to 256 frames, which does not establish long-term state correctness. Table 2 improves visibility and location, but does not directly measure identity, victim assignment, or cross-view consistency of death/respawn events. The concern is the scope of reliability that the evidence supports, rather than a demonstrated data leak or method failure.
Questions for authors
Q1: Which maps and trajectories in Table 1 and Figure 4 were entirely excluded from training, and what is the map-level split? Please provide the independent sample counts and clarify whether quantitative pose or event-consistency results exist beyond 256 frames. This would determine whether I can interpret the results as evidence of unseen-layout generalization and sustained reliability.
3 comments
Response to Reviewer 97ZB (2/2): multiplayer beyond 256 frames, events and identity, one multiplayer map
Is multiplayer state still correct beyond 256 frames?
In MultiGen, the multiplayer state does not degrade over time: every player's pose and alive/dead status live in one shared memory, updated by the same dynamics and rules for everyone, so all players always agree on it by construction. The question that remains is whether each player's rendered frames stay faithful to that shared state over long horizons, from every view.
We test exactly this over 1,768 frames. Using recorded two-player gameplay with both players moving, we write the players' positions into the shared state and generate each player's view from it. A detector trained on simulator boxes then checks whether the opponent is drawn whenever it should be visible (App. S10):
| Opponent drawn when visible (balanced accuracy) | 0–256 | 256–1k | 1k–1.8k |
|---|---|---|---|
| MultiGen | 92.4% | 96.4% | 98.0% |
Real frames: 97.8% over all frames. Rendering does not drift away from the shared state over time.
Table 2 does not measure identity, victim assignment, or cross-view death/respawn consistency
We agree that Table 2 does not. In MultiGen these events are decided once, in the shared memory: a learned kill head decides when a kill happens, a rule picks the victim, and death, respawn and identity are memory entries shared by every player. All views therefore agree on who died and who respawned by construction. What can be wrong is the events themselves.
To test them, four AI agents played 189 new deathmatches in the Doom simulator, with seeds never used in training (about 4 hours, 702 deaths). We ran the kill head on the recorded inputs and the victim rule at the true positions, and compared both with the simulator's log:
| Real deaths the kill head catches (recall, ±2 frames) | 74% |
| Kills it fires that match a real death (precision, ±2 frames) | 84% |
| Victim rule picks the true victim (chance 33%) | 90.5% |
Multiplayer training and evaluation use only one map
Yes, and Sec. 6 now states this as a limitation. The multiplayer mechanism is map-agnostic: every player's view is rendered from ray-casts of whatever map is in the shared memory, and the dynamics read the map only through those ray-casts. But our multiplayer model was trained on one map, and we have not shown that it generalizes to others. The same memory and rendering interface, trained on 100 maps for level design, does generalize to unseen layouts (first part of this reply).
We hope this clarifies what the evidence supports; if it resolves your question, we would be grateful if you would consider updating your assessment.
Thank you very much for detailed response. The addition of results for previously unseen maps, along with a more extensive analysis of pose and wall-matching, strengthens the evidence regarding layout control and provides greater transparency concerning the system's limitations. The clarifications have effectively addressed my initial concerns. Consequently, I am raising my score from 2 to 3.
Response to Reviewer 97ZB (1/2): map split, sample counts, unseen layouts, pose beyond 256 frames
Thank you for your review. We are glad you found the decomposition sensible, with a useful control interface whose practical value goes beyond producing longer videos. The responses below address each of your concerns.
Unseen-layout generalization is unresolved without a map-disjoint split
The 9 test maps of the submission (Table 1, Fig. 4) were unseen during training; the revision now states this explicitly. We also sampled 30 brand-new maps from the same generator, also never used in training, with 60 rollouts per method. The results are comparable to the original test set, and MultiGen scores best in PSNR, LPIPS and wall match over frames 129–256.
To measure whether generated frames follow the map, we trained a small probe network. From a single frame, it predicts the distance to the wall in each image column. It was trained on real game frames from 4 maps separate from all test maps, and sees only the image, never the map or the generator. Wall match is the share of image columns where the probe's distance is within 25% of the true distance ray-cast from the map, at the pose the method itself is rendering. Real frames score 0.98 (App. S8).
Q1. Which maps in Table 1 and Fig. 4 were excluded from training, and what are the sample counts?
As above, all 9 test maps of Table 1 and Fig. 4 were excluded from training, which used 100 other maps from the same generator. Each test map has one recorded episode cut into 6 chunks, so the independent sample size is 9 maps; every CI resamples maps.
Q1. Are there quantitative pose results beyond 256 frames?
Yes. MultiGen's pose error over the first 512 frames on the 30 new maps, in map units (Fig. 12 in App. S9 shows their scale on a test map):
For gameplay, what matters is not the distance to a recorded trajectory. Over long rollouts MultiGen's absolute position drifts from the recording (App. S9 reports pose error up to 3,567 frames on the original test maps), but during play no recorded trajectory exists. The player sees the generated frames and chooses the next action from them, so the only position that matters is the one the game is rendering from. What a player needs is that:
Drift from a recording is therefore a property of evaluation against logged play, not something a player would notice.