- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Artemis, a multi-agent driving world model, which generates synchronized future videos for several vehicles in a shared scene from initial observations and actions by single agents.
Artemis builds on top of existing components and combines them: specifically, starting from the set of observations assembled by the multiple agents, it reconstructs a shared static 3D point-cloud map with LingBot-Map, then renders some control maps along the planned trajectory of each agent, dividing vehicles from the background, and injects these into the pretrained Wan2.2-5B diffusion transformer through a novel GeoAdapter module, alongside a cross-agent attention layer. During autoregressive rollout, keyframes from generated videos update the shared map, and a pose memory re-initializes vehicles about to enter an agent's view.
The paper also introduces MA-CARLA, a multi-agent driving dataset rendered with CARLA engine, providing about 11K sequences with four camera views per vehicle. Experimental results on this benchmark compare Artemis with Solaris (both zero-shot and fine-tuned) and an OASIS-inspired channel-concatenation baseline. Artemis reports the best PSNR, SSIM, LPIPS, and subject consistency.
Critical strengths
I believe the main strengths of this paper are:
-
Sound and principled idea. I agree with the authors that modelling interaction between agents' observations explicitly has the potential to overcome failures of prior multi-agent frameworks solely based on implicit communication through attention. By building on estimated geometry, Artemis makes cross-view consistency a property of the conditioning signal.
-
The paper introduces a new benchmark, MA-CARLA. It is a synchronized multi-agent driving dataset with 11,283 sequences across eight towns and 17 weather presets, covering two- and three-agent interactions, moving non-agent actors, and four camera views per vehicle. This gives the field a driving-specific setting for evaluating multi-agent world models, where earlier work relied (mostly) on game scenes.
Critical weaknesses
Unfortunately, several weaknesses undermine the aforementioned strengths:
-
Comparisons with the selected baselines do not validate the main claims. The paper argues that explicit 3D state beats implicit cross-agent association, but it compares only against Solaris and an OASIS-like baseline. Neither has a 3D component. MultiWorld and ShareVerse are mentioned among related work, yet not compared or discussed. Although the authors state that recent multi-agent models are "mostly still close-sourced", this does not hold for ShareVerse and MultiWorld (both provide code and weights). Furthermore, no ablation replaces the explicit map with an implicit 3D prior either (e.g., from VGGT).
-
Results do not fully support the claims. Authors claim that Artemis gets "almost double SSIM", but this is true only against zero-shot Solaris, which is out of domain. Once fine-tuned, the gap with Solaris becomes much smaller. Furthermore, fine-tuned Solaris exposes higher background consistency (0.953 vs 0.949) and imaging quality (0.581 vs 0.568), and subject consistency differs by 0.006. In the absence of any variance study with different seeds, the improvement in subject consistency is not statistically meaningful. Concerning ablation studies: removing the static map update leaves PSNR, SSIM, and LPIPS essentially unchanged (17.7429 vs 17.7429, 0.5666 vs 0.5662, 0.3092 vs 0.3085). The long-horizon argument fundamentally builds on this update, yet clips are only 81 frames at a quite low resolution (256×416). The "separate static map" variant isn't explained in the text, and the effect sizes are small throughout.
-
The need for MA-CARLA is not clarified or demonstrated. Although ShareVerse is cited among related work, no reference is given to the dataset it proposes and releases. There is no explanation of what MA-CARLA adds with respect to it and, furthermore, there is no reference to a possible release of the MA-CARLA benchmark upon acceptance. Furthermore, it seems no ground-truth depth, poses and masks are provided, therefore with no experiments that measure how estimation errors propagate or compare with ground-truth geometry, which a simulator could supply.
Questions for authors
Answering the following questions could convince me to revise my initial assessment:
-
Comparison with MultiWorld/ShareVerse. MultiWorld uses a frozen VGGT prior for global state and is criticized in the introduction for lacking geometric cues. Why is it not compared, given that its code is public? Comparing with it, or replacing the explicit map with a VGGT-derived implicit 3D prior, is definitely necessary to test the central claim. The same applies to ShareVerse, which is very relevant to this work. Since code and models are available, it should be compared against Artemis. Please also clarify to which recent multi-agent models the statement "are mostly still close-sourced" refers.
-
Statistical reliability. Please report variance across seeds for Table 1 and Table 2. Several differences are very small, such as subject consistency (0.902 vs 0.896) and most ablation rows. Furthermore, finetuned Solaris is better on background consistency (0.953 vs 0.949) and imaging quality (0.581 vs 0.568). This should be discussed.
-
MA-CARLA contribution and release. What does MA-CARLA offer beyond the CARLA dataset released with ShareVerse (e.g., dynamic actors, three agents, motion variety)? A comparison between the two datasets is necessary. Will MA-CARLA be released, and under what terms? Furthermore, the three-agent pool is excluded from reported experiments and shown only in Fig. 8b. Quantitative comparisons would also be necessary