acceptodds
Withdrawn from ICLR 2027

Oasis-World: Toward Open-Domain Multi-Agent Interactive World Model

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

World models capable of predicting environmental dynamics conditioned on agent actions form a cornerstone of embodied intelligence and interactive simulation. Although single-agent formulations have witnessed substantial progress, extending world modeling to multi-agent environments introduces fundamental challenges including identity persistence under severe mutual occlusion, dynamic spatial tracking across co-present entities, and the grounding of concurrent behaviors to distinct visual outcomes, all of which substantially exceed the capacity of existing datasets. To bridge this data bottleneck, we present **OasisData-Engine**, an automated generation pipeline that integrates scene preparation, coordinated trajectory planning, offline photorealistic rendering, and streaming annotation to facilitate scalable parallel capture of multi-agent interactions within high-fidelity Unreal Engine environments. Leveraging this infrastructure, we establish **OasisData**, a comprehensive benchmark encompassing over 130 environments and 80 distinct character identities with configurations supporting from one to ten concurrent participants, accumulating more than 4,900 hours of video strictly paired with fine-grained frame-level action commands and structured descriptions. Furthermore, we develop **Oasis-World** as a baseline for open-domain multi-agent world modeling, conducting systematic empirical evaluations across spatial reasoning, action responsiveness, and occluded interactions to delineate critical challenges and future trajectories for interactive world modeling. **We will open source the model weights, code and data**.

Primary area:
generative models
Keywords:
interactive world model, video generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_C7qjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces OasisData-Engine and Oasis-World. The former is an automated Unreal Engine pipeline for synthesizing synchronized, action-annotated multi-agent videos. The resulting OasisData contains over 4,900 hours across 132 environments and more than 80 characters, covering configurations of one to ten agents. The latter is a Wan2.2-5B-based world model that uses per-agent action conditioning and cross-agent attention to jointly generate multiple viewpoints, training on the proposed dataset.

Critical strengths
  1. The biggest contribution is the dataset. Existing world model datasets are either egocentric navigation, or single character movement. OasisData provides practical resources in multi-agent world model area.
  2. The dataset covers satisfying amount of data, coveraging different characters and scenes. Meanwhile, the data engine is scalable, providing useful infrastructure for further data collection.
  3. Oasis-World offers a straightforward baseline and demonstrates benefits from cross-agent information exchange over independent single-agent generation.
Critical weaknesses
  1. Direct comparison with other multi-agent world model is preferable. Evaluation only compares against independent single-agent generation. MultiWorld[1] should be included for comparison.
  2. The consistency of the GPT-Image-2-generated input pairs is not established. Held-out UE scenarios with ground-truth geometry would provide more reliable spatial-consistency measurements.
  3. The dataset covers up to 10 agents, yet the evaluation only focus on 2. Scalability of agents number should be clarified.

[1] MultiWorld: Scalable Multi-Agent Multi-View Video World Models. Wu et al.

Questions for authors

Please see weakness.

Reviewer_vjtfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces OasisData-Engine, a pipeline for generating synchronized multi-agent videos based on Unreal Engine, and OasisData, a 4,959-hour dataset covering 132 environments and over 80 characters. It also presents Oasis-World, a baseline trained on OasisData. Evaluation on 100 synthetic two-agent scenarios reports lower human-rated failure rates than independently generated single-agent rollouts.

Critical strengths
  1. This paper provides a data-generation infrastructure, which combines diverse scenes and characters with synchronized multi-view videos and frame-level action annotations. This provides a potentially useful resource for studying action-conditioned multi-agent generation beyond a single game environment.
  2. The authors commit to releasing the dataset, code, and model weights. This might provide a useful foundation for further research on multi-agent world models.
Critical weaknesses
  1. Dataset quality is insufficiently validated. The paper reports neither correctness checks for the AI-generated captions nor statistics on peer visibility and interactions, leaving the reliability of its multi-agent supervision unclear.
  2. The experiment does not distinguish data and modeling contributions. The modeling setup differs, while training data and budgets are insufficiently specified, making it difficult to attribute the improvement to OasisData or joint modeling. Despite outperforming the baseline, Oasis-World still has high failure rates across all four metrics (67–92%), limiting its demonstrated reliability.
  3. Higher-agent-count data remain unvalidated. Although OasisData supports up to ten agents, training and quantitative evaluation are limited to two. The utility of data containing more than two agents is not demonstrated.
Questions for authors
  1. Were the AI-generated captions checked for correctness? What proportion of multi-agent clips contain visible peers, encounters, or occlusions?
  2. How are the structured captions incorporated into training? Which fields and training stages use them?
  3. What data were used for single-agent training, and were adaptation data and training budgets matched between the two models? Is Table 2 intended to establish dataset utility, the benefit of joint modeling, or overall system performance?
  4. Can the authors provide evidence demonstrating the utility of data with more than two agents, beyond the scalability discussion in Appendix B.2?
Reviewer_2QpzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this work authors introduces OasisData-Engine, an Unreal Engine pipeline that plans multi-agent trajectories on navigation meshes, schedules encounters between agents and renders them offline. It then pairs it with Gemini-generated structured captions. The output is OasisData consisting of 297,582 clips and 4,959 hours across 132 environments (> 80 characters). It consists of 1 to 10 agents, however majority of data has only upto 3 agents (>94% data). It also presents Oasis-World, a two-agent baseline built on Wan2.2-5B with cross-agent attention. The biggest downside is that the only quantitative evaluation is a ten-rater binary human study on 100 GPT-Image-2 two-agent scenarios against a single-agent cascade (Table 2).

Critical strengths
  • The reported scale, scene/character diversity, and the synchronized action and position annotations make OasisData potentially useful, and outshines the comparative analysis in Table 1. The pipeline combines scene preparation, encounter planning, rendering and structured captioning in a coherent collection process. The engine is described concretely enough to rebuild. This is the part of the paper the community could build on, and it is why I do not recommend clear rejection.

  • The human evaluation is run more carefully than most in this area, with rater calibration, randomized anonymized presentation and explicit failure criteria. It reports large reductions in motion-divergence and position-drift failures and leaves the remaining failures visible, which I appreciated.

Critical weaknesses
  • In my opinion, the dataset’s utility and quality validation are insufficient. Although OasisData spans 1 to 10 agents, the only model comparison is 100 generated-image two-agent scenarios with five-second rollouts. There is no held-out OasisData split,no controlled comparison that isolates what the new data contributes. Therefor the usefulness of the higher-agent-count data in particular is unmeasured. Peer visibility and occlusion coverage, which are the whole motivation of the dataset (Sec. 1), are never quantified; App. A only says that ego-view peer visibility is substantially lower.

  • The structured-caption pipeline is described extensively, but its outputs are not validated. These captions come from a long combined prompt with no audit of factual accuracy, instruction compliance, correct attribution to agents, or consistency across views, and the paper does not say whether they enter the reported model and how. Comparing the combined prompt with simpler, dimension-specific prompts would also help assess whether the additional complexity is useful.

  • The one quantitative comparison does not isolate the source of the gains, and I cannot tell which configuration produced it. The cascade generates each viewpoint independently from its own image and actions, while Oasis-World conditions on both streams and receives 30,000 further adaptation steps with roughly 3:1 multiplayer/single-player sampling. The cascade’s exact checkpoint is not stated, so the gap mixes joint modeling with extra training and OasisData exposure. Motion Divergence measures individual action adherence, so its drop from 92 to 67 does not by itself show better cross-agent conditioning.

  • The recipe is also internally inconsistent: Fig. 6 in Sec. 5.2 and Table 4 describe frozen downstream modules with only cross-agent attention trained, whereas App. B.1 says the DiT backbone receives updates. The statistics do not support the paper claim of "significantly outperforms across all metrics". The deltas (±) is undefined, aggregation across raters and scenarios is unspecified, and there is no paired uncertainty on the differences. For example, Spatial Conflict is reported as 88% ± 8% versus 76% ± 7%, but the meaning of these uncertainty terms and the paired uncertainty of the difference are unspecified.

Questions for authors
  1. How are structured captions used by the reported model? How were their factual accuracy, attribution to individual agents and consistency across synchronized views validated?
  2. Does “benchmark” refer to a held-out OasisData split, the 100-scenario OOD suite, or both?
  3. Given that 94.85% of sequences contain 1–3 agents, what fraction of multi-agent clips shows a peer for at least one second, and how frequently do occlusion/disocclusion events occur? Providing detailed breakdown and analysis of data would be nice to have.
  4. Which configuration produced Table 2: trainable modules, cross-agent attention initialization, and the exact cascade checkpoint? What are the sources and composition of the stage-1's large-scale game data, and does it overlap with OasisData?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.