- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces ME-World, a video world model that jointly generates synchronized egocentric observations of multiple interacting agents. The paper introduces a framework combining joint multi-stream denoising, view-aligned conditioning on all agents’ motions, and shared environment memory, and benchmarks with metrics for environment and identity consistency. Validation uses separate models trained on curated CoMind recordings and a newly rendered synthetic dataset, with quantitative evaluation primarily on two-agent clips. Comparisons with released baselines, ME-World achieves improvements in cross-view consistency and action alignment. Qualitative examples further demonstrate three-agent and longer autoregressive generation.
Critical strengths
-
A meaningful problem formulation beyond single egocentric agent generation. The paper targets fine-grained interactions whose actions and outcomes must agree across multiple egocentric views, rather than merely producing videos for each agent separately, which is interesting and meaningful. This is consequential because cross-view agreement is a fundamental requirement for shared-world simulation, and the formulation brings together action alignment, environment consistency, and interaction-induced updates in a concrete, evaluable setting.
-
Broad experimental coverage across datasets, baselines, and ablations. The paper evaluates on both real CoMind recordings and a substantial synthetic corpus of paired clips, with 100 paired evaluation clips in each domain. Component ablations separately examine joint generation, shared action conditioning, and shared environment memory, while evaluation covers cross-view consistency, camera and action control, and video quality. Controlled tests also validate the proposed consistency metrics. It provides complementary evidence for the framework’s effectiveness and the roles of its main components.
Critical weaknesses
- Limited novelty beyond combining established components. Joint generation is implemented through token concatenation and shared self-attention; shared action conditioning uses view-projected poses and camera-ray encodings; and environment memory combines geometric reprojection with shared reference images. These appear to be adaptations of established techniques rather than substantially new mechanisms. Although the conditions are tailored to the egocentric setting, the paper does not convincingly establish a deeper interaction-specific design.
- The main comparisons do not adequately isolate architectural gains from training and conditioning advantages. ME-World is trained on task-specific paired egocentric data and receives all agents’ motion conditions and shared observation history, whereas the main baselines use released checkpoints without task-specific fine-tuning, and several generate the two streams independently with more limited action or environment inputs. Consequently, improvements—particularly in counterpart action alignment and cross-view consistency—may partly reflect additional information and task adaptation rather than superior modeling. The same-backbone, same-data MultiWorld adaptation and internal ablations mitigate this concern, but do not replace a strong independently generated baseline trained with the same data and supplied with equivalent conditions.
Questions for authors
What I am mainly concerned about is the novelty of the framework and its scalability going forward. I will consider whether to revise my score based on the other reviewers' opinions and the authors' responses in the rebuttal.