acceptodds
Under review as a conference paper at ICLR 2027

MI-World: Multi-Inhabitant World Models with 3D Evolution

Abstract

Interactive world models simulate future environments in response to user actions. Current approaches rely primarily on video generation, where visual impacts often appear correct while the underlying physical state remains unchanged. This disconnect becomes disruptive when multiple characters share the scene. A removed block can reappear once a camera turns away, character health can remain static despite visible hits, and off-screen non-player characters leave no lasting traces for players who arrive later. We present **MI-World**, an interactive world model that couples multi-character simulation with an evolving 3D voxel map and structured entity records. Scene completion synthesizes unobserved terrain without overwriting established coordinates, preserving earlier structures during exploration. Player and NPC controls update voxel occupancy and character attributes through a shared transition model, enabling autonomous NPCs to modify the environment even while outside any active viewpoint. Perspective video streams are then rendered directly from the updated canonical state. We collect a dataset of 45,572 synchronized multi-character trajectories annotated with explicit interaction events. MI-World achieves a cross-view consistency FVD of 240.7, compared with 280.0 for γ-World and 443.1 for Solaris. Under recorded controls, it recovers reference block edits and attacks at F1 scores of 0.886 and 0.851, respectively. The demo is at [http://miworld-anonymous.github.io/](http://miworld-anonymous.github.io/).

Primary area:
generative models
Keywords:
Interactive Multi-Agent World Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_29b2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

MI-World aims to let multiple characters inhabit a generated world in which their actions leave lasting, shared consequences. This work connects terrain generation, action prediction, and visual rendering through a persistent voxel map and character states, allowing block edits and interactions to affect subsequent observations. The system also incorporates autonomous NPCs through the same interaction interface. Experiments in a Minecraft-like environment evaluate visual generation and the accuracy of state updates under recorded multi-character controls.

Critical strengths

Its strongest contribution is the coherent integration for persistent multi-character worlds. MI-World connects terrain expansion, action-driven voxel and entity updates, and view generation through a shared 3D state, with NPCs using the same transition interface. Separating unknown-region completion from intentional state changes provides a clear design for retaining interaction outcomes during exploration. The integration offers a useful architectural reference, supported by initial event- and state-prediction results under supplied controls.

Critical weaknesses
  1. The experiments support improved visual quality and state prediction under recorded controls, but leave important parts of the shared-world claim untested. FVD/FID do not directly measure whether different characters correctly observe the same persistent edit, while supplying recorded NPC actions leaves autonomous closed-loop behavior without quantitative evaluation. State accuracy is measured up to 64 steps; the 600-step rollouts remain qualitative. These gaps limit the evidence for reliable interaction across characters, viewpoints, and extended rollouts.

  2. The benefit of this particular integration is difficult to assess without closer comparisons or ablations. Comparisons with video-based models show useful improvements, but do not establish what MI-World adds beyond existing explicit/shared-state approaches. Targeted comparisons or component ablations would help identify which design choices account for the gains.

Questions for authors
  1. Have you measured whether the same edit is correctly reflected in different characters’ views, including after a delayed revisit? It would be more clear if it clarifies how many trajectories were evaluated and how they were selected.

  2. Could a comparison with a closely related method, or targeted ablations, show which design choices account for the gains?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 29b2 (1/2)

We thank Reviewer 29b2 for the careful review. We have added cross-character edit and revisit evaluations, closed-loop NPC tasks, a PERSIST comparison, and component ablations to address the raised concerns.


 

W1 / Q1: Cross-Character Edits, Delayed Revisits, and Autonomous Behavior

(1) Cross-Character Edit Observation and Delayed Revisits

To evaluate whether the same block edit remains correctly visible across character views, we design two experiments:

  • Delayed look-back: Character A places a block, and both characters observe it before turning away. They then look back at the same location after 64, 256, or 512 frames, with 12, 8, and 10 scene groups for these three settings, respectively (30 groups in total).
  • Spatial revisit: After both characters observe the placed block, character B walks away and returns, while character A turns away before the final observation. We test walking distances of 12 and 16 blocks, with 15 scene groups for each distance (30 groups in total).

Each scene group contains two paired two-character trajectories: one with block placement and one no-edit control that follows the same actions except for the placement. Thus, the two experiments comprise 60 scene groups and 120 two-character trajectories in total. Scenes are selected using predefined trajectory criteria and engine-verified events, rather than generated-video quality.

A VLM evaluates whether the edit is correctly reflected in both characters' views. At each observation phase (immediate or revisit), we evaluate 8 judgments per scene group: 2 characters × 2 temporal probes × 2 conditions (placement/no-edit). We report observation accuracy (the proportion of correct individual judgments) and group success rate (all eight judgments correct). Results are means ± standard deviations over three judging runs (%).

ExperimentPhaseObservation Accuracy ↑Group Success Rate ↑
Delayed look-backImmediate94.03 ± 0.2078.89 ± 1.57
Delayed look-backRevisit83.89 ± 0.7153.33 ± 2.72
Spatial revisitImmediate95.97 ± 0.3984.44 ± 1.57
Spatial revisitRevisit86.11 ± 0.3953.33 ± 0.00

(2) Autonomous NPC Behavior under Generated Observations

We evaluate a single NPC in closed loop: after initialization, it selects actions from its own generated views, the Local World Writer updates the world state, and the World Generator produces subsequent observations.

We evaluate each task on 100 test trajectories.

TaskSpecificationSuccess Rate ↑
Straight-line building2-block line55%
Wall repairFill a 2-block gap42%
Wall construction2×22\times2 vertical wall36%
Floor construction2×22\times2 horizontal surface21%
Attack targetSpecified hostile target44%

All results are judged by a VLM from the task instruction and full generated video. For building tasks, success means completing the target shape within the action budget; for the attack task, success means performing a target-directed attack.

These results demonstrate that the NPC can perform certain short-horizon text-driven tasks in a closed loop using its own generated observations.

AuthorsOpenReview ↗

Response to Reviewer 29b2 (2/2)

W2 / Q2: Explicit-State Comparison and Component Ablations

(1) Direct Comparison with PERSIST

We agree that comparisons with video world models alone make it difficult to separate the benefits of explicit 3D state from those of specific design choices. PERSIST also uses persistent 3D state and is therefore a relevant reference. The original submission focused on multi-character interactions, whereas PERSIST targets single-character settings and requires substantial adaptation and training to extend to multiple characters. We therefore add a direct comparison on PERSIST's single-character benchmark.

We fine-tune MI-World's Global World Writer, Local World Writer, and World Generator on PERSIST's training data, setting absent multiplayer conditions to neutral defaults. Both methods use the same 168 evaluation trajectories, action inputs, resolution, and prediction horizons. FVD is computed on non-overlapping 16-frame clips.

MethodFVD@200 ↓FVD@400 ↓FVD@600 ↓FID@200 ↓FID@400 ↓FID@600 ↓
PERSIST13114415234.535.336.0
MI-World12413614432.833.734.8

MI-World achieves lower FVD/FID at all three horizons, providing a direct comparison with an explicit-state method. Both methods use explicit 3D state, while MI-World separates unknown-region completion from interaction-driven local updates. The ablations below further examine state updates and known-voxel protection.

 

(2) Targeted Component Ablations

We conduct inference-time ablations on the same 100 two-character test cases, each with two views over 384 frames. All settings use the same recorded actions and model weights.

SettingFVD ↓FID ↓LPIPS ↓PSNR ↑SSIM ↑
Masked World Generator voxel conditioning1896.05165.270.805010.360.5758
Frozen world-state updates391.4837.090.544414.080.5565
Global World Writer overwrites known voxels581.4460.610.595713.760.5788
Full MI-World182.6223.320.366418.560.6651
World Generator with True States43.7814.830.188022.750.7511

All three interventions reduce generation quality, indicating the importance of voxel conditioning, ongoing state updates, and known-voxel protection in the trained system. Freezing state updates and allowing known-voxel overwrites increase FVD from 182.62 to 391.48 and 581.44, respectively, supporting the design of updating interaction outcomes while protecting established regions. The World Generator with True States further indicates substantial room for improvement through more accurate world-state prediction.

Thank you for your clear response and explanation. New experiments look good to me. I am happy to raise my overall rating.

Thank you for your time and understanding. We sincerely appreciate your constructive feedback and valuable suggestions, which have helped us improve our work.

Reviewer_vB6POpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces an interactive world model that world model that grounds multi-character video simulation in a persistent, shared 3D state consisting of a voxel map, a known-coordinates mask, character records and interaction logs. The architecture utilises three components: (i) a Global World Writer which synthesises unobserved terrain while preserving previously established coordinates and edits (ii) Local World Writer that processes multi-agent controls to predict camera poses and interaction events and (iii) World Generator that renders the video streams from the projected 3D state. To train and evaluate the model, the authors introduce a large-scale dataset collected in Luanti, consisting of 45,572 synchronised multi-character trajectories with event attribution. Experiments demonstrate that their work outperforms existing multiplayer baselines on FVD and FID across five categories

Critical strengths
  • Explicitly distinguishing between unknown coordinates and known air resulting from block removal prevents the Global World Writer from overwriting player edits during return visits. This directly resolves the "reappearing block" failure mode common in video world models.
  • Completion, transitions and rendering are handled separately. Together with the deterministic Apply operator and the interaction log, this makes state changes auditable and errors traceable.
  • Providing 45,572 multi-agent trajectories across 14 diverse scenarios with explicit ground-truth event attributions (acting character, target, coordinate, operation, and health delta) is a valuable benchmark contribution for the community.
Critical weaknesses
  • The support for NPCs is a central component of this work. However, experiments evaluate how the world responds to actions, not whether the NPC policy selects useful actions or remains effective when conditioned on its own generated observations.
  • In Equations 3 and 5, the Global World Writer is prevented from overwriting established coordinates via the deterministic mask. While this guarantees by construction that exploratory completion cannot erase prior structures or "known air," however, if a mistake happens it is never repaired, and it also can be damaged further by completion. Further discussion or an ablation of that scenario is needed.
  • The paper acknowledges persistent-state and shared-state predecessors, including PERSIST. The comparisons focus on video-based multiplayer baselines which makes it difficult to determine how much of the improvement follows from introducing an explicit shared state or how much depends on this work's particular decomposition and learnt transition model.
  • FVD and FID measure the distributional similarity of generated samples to real data, so views that disagree with each other can still score well. The manuscript partly acknowledges this, yet it claims "improved cross-view consistency" on this basis. A a metric that compares views directly, such as VLM-judge questions as in Solaris, would strengthen these claims.
  • Performance improvement cannot yet be attributed to specific design choices. The most important ablation is the renderer without the state projection, i.e., the same backbone used as a video-only model. This would isolate the value of the explicit state.
Questions for authors

See "Weaknesses" section. I'm happy to increase the score if those are addressed adequately.

4 comments
AuthorsOpenReview ↗

Response to Reviewer vB6P (1/2)

We thank Reviewer vB6P for recognizing our known-state protection mechanism, modular design, and dataset contribution. We have added closed-loop NPC evaluations, a PERSIST comparison, component interventions, and direct cross-view evaluations, and clarified the limitations of deterministic masking.


 

W1: Autonomous NPC Behavior under Generated Observations

We agree that action-response quality does not establish NPC policy effectiveness. We therefore evaluate a single NPC in closed loop: after initialization, it selects actions from its own generated views, the Local World Writer updates the state, and the World Generator produces subsequent observations. No recorded NPC actions or assistance from other players are used.

We evaluate each task on 100 test trajectories.

TaskSpecificationSuccess Rate ↑
Straight-line building2-block line55%
Wall repairFill a 2-block gap42%
Wall construction2×22\times2 vertical wall36%
Floor construction2×22\times2 horizontal surface21%
Attack targetSpecified hostile target44%

A VLM (gpt-6.1-sol) judges the task instruction and full generated video. For building tasks, success means completing the target shape within the action budget; for the attack task, success means performing a target-directed attack.

These results demonstrate that the NPC can perform certain short-horizon text-driven tasks in a closed loop using its own generated observations.


 

W2: Deterministic Masking and Error Correction

We agree that deterministic masking introduces a trade-off between state persistence and error correction. MI-World prioritizes preserving established world states, preventing exploratory completion from unintentionally overwriting existing structures or known air.

However, errors introduced during initial completion or interaction prediction may persist and affect subsequent world expansion. Addressing this requires a mechanism that selectively corrects erroneous states while preserving intentional edits. We will clarify this design trade-off in the revised paper and discuss observation-driven state correction as a future direction.

AuthorsOpenReview ↗

Response to Reviewer vB6P (2/2)

W3: Comparison with PERSIST

The original evaluation prioritized multi-character models. We agree that PERSIST, a closely related explicit-state method, should also be compared, and have added experiments on its single-character benchmark.

We fine-tune the Global World Writer, Local World Writer, and World Generator on PERSIST's training data, setting absent multiplayer-specific conditions to neutral defaults. Both methods use the same 168 evaluation trajectories, action inputs, resolution, and prediction horizons. FVD is computed on non-overlapping 16-frame clips.

MethodFVD@200 ↓FVD@400 ↓FVD@600 ↓FID@200 ↓FID@400 ↓FID@600 ↓
PERSIST13114415234.535.336.0
MI-World12413614432.833.734.8

MI-World achieves lower FVD/FID at all horizons, demonstrating improvements over a closely related explicit-state world model.


 

W4: Cross-View Consistency Evaluation

We agree that FVD/FID cannot directly establish cross-view consistency. Drawing on Solaris's VLM question design, we evaluate Movement, Grounding, Memory, Building, and Consistency. Consistency jointly presents two synchronized views to the judge to assess whether their scene relationship matches the expected answer.

All methods use gpt-6.1-sol and exact-match query accuracy. We report percentages as mean ± standard deviation over three repeated judging runs.

MethodMovement ↑Grounding ↑Memory ↑Building ↑Consistency ↑
Solaris72.92 ± 0.8575.00 ± 0.0063.28 ± 0.0044.79 ± 1.4768.65 ± 2.66
Gamma-World60.07 ± 0.4983.75 ± 0.0070.61 ± 0.3768.75 ± 2.5545.83 ± 1.95
MI-World67.32 ± 1.3080.81 ± 1.2877.25 ± 1.1069.67 ± 5.3182.44 ± 2.97

MI-World achieves 82.44% Consistency and 77.25% Memory. These results provide direct evidence of cross-view semantic consistency in the evaluated scenarios. We will distinguish this evidence from visual-quality metrics in our revised claims.


 

W5: Component Ablations and the Role of Explicit State

We conduct inference-time interventions on the same 100 two-character cases, each with two views over 384 frames. All settings use the same recorded actions and model weights, without additional training.

SettingFVD ↓FID ↓LPIPS ↓PSNR ↑SSIM ↑
Masked voxel conditioning1896.05165.270.805010.360.5758
Frozen world-state updates391.4837.090.544414.080.5565
Global World Writer overwrites known voxels581.4460.610.595713.760.5788
Full MI-World182.6223.320.366418.560.6651
World Generator with True States43.7814.830.188022.750.7511

All three interventions reduce generation quality, indicating the importance of voxel conditioning, ongoing state updates, and known-voxel protection in the trained system. The World Generator with True States uses ground truth for all state inputs, indicating further room for improvement through more accurate state prediction.

I appreciate the response and explanations. The new experiments address my concerns and I'm happy to raise my score.

Thank you for your positive feedback. We also appreciate your constructive feedback and valuable suggestions on our work.

Reviewer_fp77OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors introduce a framework for world models where several characters share a persistent world state. The state includes a 3D voxel map (what kind of block occupies each cell), characters poses and attributes. The main components of the system include: (1) Global world writer - which fills in new regions that were never observed before, (2) Local world writer - which takes all actions of all characters and predicts where each character moves and looks, and (3) World generator - which generates each character view from the updated state.

In addition, the authors introduce a new dataset of synchronized trajectories with 2-4 characters and labelled interaction events. On this dataset their model outperforms some baselines on standard metrics.

Critical strengths
  1. The paper is generally clear and well organized.
  2. The new dataset can be used for further research and direct comparisons.
  3. The experimental results show improvements over the evaluated baselines.
Critical weaknesses
  1. The paper re-uses several components from the PERSIST method (Garcin et al. 2026), but does not compare to it in experiments.
  2. There are no component ablations.

I am happy to increase my score if the authors can address these points.

Questions for authors
  1. Could you clarify why PERSIST was not included in the comparisons ?
  2. On line 224 why is it 13 (i.e. radius of 6) ? How sensitive are the results to the window size ?
3 comments
AuthorsOpenReview ↗

Response to Reviewer fp77

We thank Reviewer fp77 for recognizing the paper's clarity, the value of our dataset, and the experimental improvements. We have added a comparison with PERSIST, component ablations, and a local-window sensitivity analysis, as detailed below.


 

W1 / Q1: Comparison with PERSIST

The original submission focused on multi-character interactions and therefore prioritized world models that directly support multi-character generation. PERSIST, as a closely related explicit-state method, should also be included in the comparisons. We have therefore added experiments evaluating MI-World on PERSIST's single-character benchmark.

We fine-tune MI-World's Global World Writer, Local World Writer, and World Generator on PERSIST's training data, setting multiplayer-specific conditions absent from the data to neutral defaults. Both methods are evaluated on the same 168 evaluation trajectories, with identical action inputs, resolution, and prediction horizons.

MethodFVD@200 ↓FVD@400 ↓FVD@600 ↓FID@200 ↓FID@400 ↓FID@600 ↓
PERSIST13114415234.535.336.0
MI-World12413614432.833.734.8

In this single-character setting, MI-World achieves lower FVD and FID at all three prediction horizons. This improvement highlights the benefit of MI-World's separation of world completion and interaction-driven state updates in maintaining persistent world consistency.


 

W2: Component Ablations

We have added component ablations to analyze how the core modules and their design mechanisms affect generation quality.

We conduct inference-time ablations on the same 100 two-character test cases, each containing two views and 384 frames. All settings share the same model weights without additional training. Specifically, we remove voxel conditioning from the World Generator, freeze world-state updates, or allow the Global World Writer to overwrite known voxels, to examine the model's dependence on these mechanisms.

SettingFVD ↓FID ↓LPIPS ↓PSNR ↑SSIM ↑
World Generator without voxel conditioning1896.05165.270.805010.360.5758
Frozen world-state updates391.4837.090.544414.080.5565
Global World Writer overwrites known voxels581.4460.610.595713.760.5788
Full MI-World182.6223.320.366418.560.6651
World Generator with True States43.7814.830.188022.750.7511

All three interventions reduce generation quality, indicating that voxel conditioning, ongoing state updates, and known-voxel protection each play an important role in the trained system. Additionally, the results of the World Generator with True States indicate substantial room for improvement through more accurate world-state prediction.


 

Q2: Local Window Size and Sensitivity

We use a cubic window with side length 2r+1=132r+1=13 and radius r=6r=6 to cover both direct interaction locations and the surrounding context needed for motion prediction. We measure edit-location coverage over 9,763 block-edit events from 1,000 multi-character validation trajectories. We also conduct a short adaptation experiment: all window variants start from the same Local World Writer checkpoint, undergo 200 update steps using the same training samples, and are evaluated on the same validation samples.

Window SizeEdit-Location CoverageEvent F1 ↑Endpoint Position Error ↓
737^374.18%——
939^399.65%0.4250.195
11311^3100.00%0.6200.199
13313^3 (default)100.00%0.6400.218
15315^3100.00%0.6590.259

The results show that an 11311^3 window already covers all edit locations in the evaluated samples, and the 11311^3, 13313^3, and 15315^3 variants all achieve good prediction performance. As the window size increases, event F1 gradually improves, while endpoint position error also increases. We adopt 13313^3 as the default to balance these two metrics.

Reviewer_fp77OpenReview ↗

Thanks

Thank you very much for these new results. I am happy to increase my score.

Thank you for your time and understanding. We also appreciate your constructive feedback and valuable suggestions on our work.

Reviewer_ZTwTOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MI-World, a multi-inhabitant world model built around a shared 3D voxel map and structured entity states. The framework separates world expansion, action-driven state updates, and state-conditioned video generation, while integrating autonomous NPCs into the same interaction loop. The authors introduce a dataset of 45,572 synchronized multi-character trajectories and evaluate video quality, interaction prediction, and accumulated state errors in a block-world environment.

Critical strengths

The paper addresses an important limitation of video-based world models: preserving interaction consequences across viewpoints and time. Separating unknown-region completion from deliberate state edits provides a clear mechanism for retaining environmental changes, including removed blocks. The unified update interface for players and NPCs is also a coherent design for persistent multi-character interaction. Additionally, the dataset’s explicit event annotations and evaluations of block edits, attacks, and continuous state rollouts complement visual-quality metrics with more direct measures of interaction correctness.

Critical weaknesses

1.The proposed persistent consistency relies on explicit block-category maps, entity attributes, and discrete interaction definitions. These design choices are closely aligned with the evaluation environment, but their applicability to real-world scenes involving continuous motion and non-rigid dynamics remains unclear. Moreover, in highly discrete and structured environments such as Minecraft-style block worlds, conventional rendering is a straightforward alternative when geometry and the relevant visual assets are available. The paper does not sufficiently justify the need for a flow-matching-based generator in this setting. Although Gamma-World and Solaris also use flow matching to generate Minecraft videos, they do not require an explicit scene-state representation of this kind, making their architectures less tied to the particular environment. In comparison, MI-World’s dependence on its domain-specific representation and update mechanisms raises concerns about its broader applicability. 2.The paper does not provide sufficient details about how the structured world representation conditions the World Generator. Although it states that projected features are patch-embedded together with noisy image latents, the nature of these features and the fusion mechanism remain unspecified. Are the “projected features” RGB proxy frames or videos, semantic or depth maps, or learned feature maps? How exactly are they combined with the noisy frame latents? These omissions make it difficult to reproduce the method and assess whether the World Generator introduces any additional technical novelty beyond existing approaches. 3.The paper does not report MI-World’s computational efficiency or provide sufficient details about its inference configuration. Gamma-World and Solaris employs DMD-based few-step distillation and reports real-time generation. In contrast, MI-World does not specify whether distillation is used or how many sampling steps are required. If MI-World is evaluated with substantially more sampling steps than the distilled baselines, its reported quality improvements may come at a higher cost. 4.Minor weakness: "State tokensx" should be "State tokens" in Fig. 2.

Questions for authors

See Critical Weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer ZTwT

We sincerely thank Reviewer ZTwT for the positive assessment and constructive feedback. We appreciate the recognition of our approach to persistent multi-character interaction. We address the concerns point by point below.


W1: Scope of the Representation and Necessity of Generative Rendering

We agree that our voxel-based representation and discrete interaction mechanisms are specifically designed for Minecraft-like environments and do not directly extend to continuous motion or non-rigid dynamics. Our goal is not to propose a universal state representation for all physical environments. We acknowledge this limitation and view broader generalization as an important direction for future work.

We also agree that conventional rendering is a straightforward choice when complete visual assets and a rendering engine are available. However, our explicit state only maintains essential structural information, such as geometry, block categories, and resident attributes, without requiring complete textures, materials, or rendering assets. The Flow Matching generator learns to map these structured conditions to visual observations, synthesizing visual details not explicitly represented in the state. Thus, explicit state maintenance and generative rendering serve complementary roles: the former preserves interaction consequences, while the latter generates corresponding visual observations. Our work focuses on combining these two components to achieve persistent consistency in multi-character worlds.


W2: Projected Features and Conditioning Mechanism

We thank the reviewer for pointing out the lack of clarity in our description. The "projected features" in Fig. 2 are learned feature maps derived from world and resident states and aligned with the target viewpoint, rather than RGB proxy images.

For world voxels, block categories are mapped to learnable embeddings and projected into the target camera view. The per-pixel depth-layer features are augmented with depth encodings and compressed into 2D feature maps using 1D convolutions along the depth dimension. For other residents, we project their 3D positions into the corresponding regions of the target view and encode spatial features incorporating their orientations, actions, and appearance information.

At the input fusion stage illustrated in Fig. 2, these spatial conditioning features are aligned with the noisy video latents at the same spatial resolution, concatenated along the channel dimension, and jointly patch-embedded using a shared strided 2D convolution to produce visual tokens for the Transformer. We will clarify these details and update the corresponding annotations in Fig. 2.


W3: Distillation and Inference Efficiency

MI-World adopts Self-Forcing with a DMD objective to distill the causal Flow Matching renderer into a 4-step generator. We use KV caching for streaming autoregressive generation. The renderer contains 0.5B parameters.

On a single NVIDIA H20 GPU, MI-World achieves approximately 13 FPS when simultaneously generating two views at 640×360 resolution. We will include these inference configurations and efficiency measurements in the appendix of the revised paper.


W4: Typo in Fig. 2

Thank you for pointing this out. We will correct "State tokensx" to "State tokens" in Fig. 2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Our Response

Dear Area Chair,

Thank you in advance for your hard work. We have carefully addressed each point raised by the reviewers and have submitted detailed point-by-point responses.

We appreciate the reviewers’ recognition of the following strengths:

  • Important problem and coherent framework. Reviewer ZTwT recognized that MI-World addresses “an important limitation of video-based world models,” while Reviewer 29b2 highlighted its “coherent integration for persistent multi-character worlds.”
  • Persistent and traceable interaction outcomes. Reviewer vB6P appreciated our mechanism for preserving player edits during revisits and noted that the modular design “makes state changes auditable and errors traceable.”
  • Valuable dataset and benchmark. Reviewers ZTwT, fp77, and vB6P recognized the value of our dataset of 45,572 synchronized multi-character trajectories with explicit interaction annotations, with vB6P describing it as “a valuable benchmark contribution for the community.”
  • Empirical improvements and clear presentation. Reviewer fp77 acknowledged the improvements over the evaluated baselines and found the paper “generally clear and well organized.”

During the rebuttal, we added the following experiments and clarifications to address the reviewers’ main concerns:

  • Baseline comparison and component ablations. MI-World achieves lower FVD and FID than PERSIST at all evaluated horizons from 200 to 600 frames. Inference-time ablations further support the importance of voxel conditioning, continuous state updates, and known-voxel protection.

  • Cross-view consistency and persistence. Direct VLM-based evaluation shows that MI-World achieves 82.44% cross-view consistency, compared with 68.65% for Solaris and 45.83% for Gamma-World. Additional cross-character edit and revisit experiments demonstrate that interaction outcomes remain observable across characters and after delayed or spatial revisits.

  • Closed-loop NPCs and implementation clarifications. We added five NPC tasks using the NPC’s own generated observations, demonstrating short-horizon task completion without recorded NPC actions. We also clarified the conditioning mechanism, inference efficiency, and limitations, including the trade-off between state persistence and error correction.

Following our responses and additional experiments, Reviewers fp77, vB6P, and 29b2 raised their ratings. All four reviewers now assign a rating of 3 (Weak Accept). In particular, Reviewer vB6P confirmed that “The new experiments address my concerns,” while Reviewers fp77 and 29b2 also provided positive feedback on the additional results.

We will incorporate all new results, clarifications, and limitations into the revised manuscript. We sincerely appreciate the reviewers’ constructive feedback and thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.