- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
MI-World aims to let multiple characters inhabit a generated world in which their actions leave lasting, shared consequences. This work connects terrain generation, action prediction, and visual rendering through a persistent voxel map and character states, allowing block edits and interactions to affect subsequent observations. The system also incorporates autonomous NPCs through the same interaction interface. Experiments in a Minecraft-like environment evaluate visual generation and the accuracy of state updates under recorded multi-character controls.
Critical strengths
Its strongest contribution is the coherent integration for persistent multi-character worlds. MI-World connects terrain expansion, action-driven voxel and entity updates, and view generation through a shared 3D state, with NPCs using the same transition interface. Separating unknown-region completion from intentional state changes provides a clear design for retaining interaction outcomes during exploration. The integration offers a useful architectural reference, supported by initial event- and state-prediction results under supplied controls.
Critical weaknesses
-
The experiments support improved visual quality and state prediction under recorded controls, but leave important parts of the shared-world claim untested. FVD/FID do not directly measure whether different characters correctly observe the same persistent edit, while supplying recorded NPC actions leaves autonomous closed-loop behavior without quantitative evaluation. State accuracy is measured up to 64 steps; the 600-step rollouts remain qualitative. These gaps limit the evidence for reliable interaction across characters, viewpoints, and extended rollouts.
-
The benefit of this particular integration is difficult to assess without closer comparisons or ablations. Comparisons with video-based models show useful improvements, but do not establish what MI-World adds beyond existing explicit/shared-state approaches. Targeted comparisons or component ablations would help identify which design choices account for the gains.
Questions for authors
-
Have you measured whether the same edit is correctly reflected in different characters’ views, including after a delayed revisit? It would be more clear if it clarifies how many trajectories were evaluated and how they were selected.
-
Could a comparison with a closely related method, or targeted ablations, show which design choices account for the gains?
4 comments
Response to Reviewer 29b2 (2/2)
W2 / Q2: Explicit-State Comparison and Component Ablations
(1) Direct Comparison with PERSIST
We agree that comparisons with video world models alone make it difficult to separate the benefits of explicit 3D state from those of specific design choices. PERSIST also uses persistent 3D state and is therefore a relevant reference. The original submission focused on multi-character interactions, whereas PERSIST targets single-character settings and requires substantial adaptation and training to extend to multiple characters. We therefore add a direct comparison on PERSIST's single-character benchmark.
We fine-tune MI-World's Global World Writer, Local World Writer, and World Generator on PERSIST's training data, setting absent multiplayer conditions to neutral defaults. Both methods use the same 168 evaluation trajectories, action inputs, resolution, and prediction horizons. FVD is computed on non-overlapping 16-frame clips.
| Method | FVD@200 ↓ | FVD@400 ↓ | FVD@600 ↓ | FID@200 ↓ | FID@400 ↓ | FID@600 ↓ |
|---|---|---|---|---|---|---|
| PERSIST | 131 | 144 | 152 | 34.5 | 35.3 | 36.0 |
| MI-World | 124 | 136 | 144 | 32.8 | 33.7 | 34.8 |
MI-World achieves lower FVD/FID at all three horizons, providing a direct comparison with an explicit-state method. Both methods use explicit 3D state, while MI-World separates unknown-region completion from interaction-driven local updates. The ablations below further examine state updates and known-voxel protection.
(2) Targeted Component Ablations
We conduct inference-time ablations on the same 100 two-character test cases, each with two views over 384 frames. All settings use the same recorded actions and model weights.
| Setting | FVD ↓ | FID ↓ | LPIPS ↓ | PSNR ↑ | SSIM ↑ |
|---|---|---|---|---|---|
| Masked World Generator voxel conditioning | 1896.05 | 165.27 | 0.8050 | 10.36 | 0.5758 |
| Frozen world-state updates | 391.48 | 37.09 | 0.5444 | 14.08 | 0.5565 |
| Global World Writer overwrites known voxels | 581.44 | 60.61 | 0.5957 | 13.76 | 0.5788 |
| Full MI-World | 182.62 | 23.32 | 0.3664 | 18.56 | 0.6651 |
| World Generator with True States | 43.78 | 14.83 | 0.1880 | 22.75 | 0.7511 |
All three interventions reduce generation quality, indicating the importance of voxel conditioning, ongoing state updates, and known-voxel protection in the trained system. Freezing state updates and allowing known-voxel overwrites increase FVD from 182.62 to 391.48 and 581.44, respectively, supporting the design of updating interaction outcomes while protecting established regions. The World Generator with True States further indicates substantial room for improvement through more accurate world-state prediction.
Thank you for your clear response and explanation. New experiments look good to me. I am happy to raise my overall rating.
Thank you for your time and understanding. We sincerely appreciate your constructive feedback and valuable suggestions, which have helped us improve our work.
Response to Reviewer 29b2 (1/2)
We thank Reviewer 29b2 for the careful review. We have added cross-character edit and revisit evaluations, closed-loop NPC tasks, a PERSIST comparison, and component ablations to address the raised concerns.
W1 / Q1: Cross-Character Edits, Delayed Revisits, and Autonomous Behavior
(1) Cross-Character Edit Observation and Delayed Revisits
To evaluate whether the same block edit remains correctly visible across character views, we design two experiments:
Each scene group contains two paired two-character trajectories: one with block placement and one no-edit control that follows the same actions except for the placement. Thus, the two experiments comprise 60 scene groups and 120 two-character trajectories in total. Scenes are selected using predefined trajectory criteria and engine-verified events, rather than generated-video quality.
A VLM evaluates whether the edit is correctly reflected in both characters' views. At each observation phase (immediate or revisit), we evaluate 8 judgments per scene group: 2 characters × 2 temporal probes × 2 conditions (placement/no-edit). We report observation accuracy (the proportion of correct individual judgments) and group success rate (all eight judgments correct). Results are means ± standard deviations over three judging runs (%).
(2) Autonomous NPC Behavior under Generated Observations
We evaluate a single NPC in closed loop: after initialization, it selects actions from its own generated views, the Local World Writer updates the world state, and the World Generator produces subsequent observations.
We evaluate each task on 100 test trajectories.
All results are judged by a VLM from the task instruction and full generated video. For building tasks, success means completing the target shape within the action budget; for the attack task, success means performing a target-directed attack.
These results demonstrate that the NPC can perform certain short-horizon text-driven tasks in a closed loop using its own generated observations.