acceptodds
Under review as a conference paper at ICLR 2027

Share Poses, Not Pixels: An Interventional Study of Shared World Models for Multi-Agent Navigation

Abstract

A navigation world model predicts what an agent will see after acting. It is natural to ask whether several agents could share one, each broadcasting its intent so that every agent's imagined future contains the others, and whether such a shared model could serve as a negotiation device, scoring one's own plans by the reactions they provoke. We test this with a conditional diffusion world model in the style of Navigation World Models, extended with a peer channel, inside a simulator in which interaction strength is a dial and a control world with exactly zero interaction exists for every scene. (i) The model learns to use a broadcast pose: a matched difference-in-differences against the control puts the broadcast at -0.014 LPIPS (95% CI [-0.017,-0.012]), five to eight times its value where nothing interacts. (ii) In closed-loop decentralised planning against neighbours of unknown disposition, reading the neighbour out of the imagined frame is no better than ignoring it (46.7% vs. 48.9% collisions) while a six-bit broadcast of its declared pose matches an oracle (2.2%), and the gap widens from 17 to 67 points as the crowd grows from 2 to 5. A ladder of oracles attributes this to field of view, the pixel readout, and the imagination, which draws a neighbour within one metre at 0.42 of its size; a readout fitted to the model's own frames does not recover it. (iii) A state head trained jointly on the same trunk does: it places the neighbour 3.2 s ahead to 0.15 m in one forward pass and learns how the neighbour reacts to the ego's action (slope 0.39 against 0.00 in the control world), a dependence the pixel objective never acquires. Yet in closed loop this head collides nearly three times as often as constant-velocity extrapolation from the same input, because what it learned is the training population's policy and the neighbour in front of it may not share it. Pixels lose the number; a learned model of others must bet on their policy; a declared intent does neither. In this setting, sharing should happen at the level of poses and intents rather than pixels.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, multi-agent navigation, diffusion models, decentralized planning, communication, interventional evaluation, simulation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zwxnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies the benefits of a shared world-model in a multi-agent reinforcement learning setting. The authors aim to answer whether a shared world model could serve as a negotiation mechanism.

Critical strengths

Assessing multi-agent interaction with world models is a timely problem.

Critical weaknesses
  • the writing of the paper is confusing at times, e.g., "4 channels" (l.146) and "a six-dimensional channel" (l.147): why not just call the latter "6 channels" which is much cleaner and shorter. Because of the authors use of "a six-dimensional channel" I don't even know exactly whether what I think they are referring to ("6 channels") is correct.
  • the submission remains unclear at times, e.g., "a six-dimensional channel" (l.147) contains 1) neighbor's position and 2) heading as well as 3) declared future position and 4) declared future heading. How do these 4 aspects amount to a six-dimensional channel? This wasn't explained in the main paper.
  • the submission uses terminology that isn't well introduced, e.g., "two arms differ only in this channel" (l.148): the concept of an arm hasn't been introduced prior to l.148 and when reading this sentence I have no idea what the authors are referring to. A search for "arm" reveals only one prior occurrence in l.82 which also doesn't explain its meaning in the paper's context
  • similarly, the "why not recorded data" section (l.153ff) continues to assume that I know the environment which a reader may not. The following paragraph then describes a simulator but doesn't explain how this simulator relates to the environment. At least I was not able to understand this.
  • Why is LPIPS a useful metric here? No motivation is provided in l.203. Only later when reading l.236 did I understand that the authors are trying to assess planning quality?
  • A single synthetic environment seems to be investigated which makes it hard to judge generality.

Overall, in its current form the paper is not very well organized and the main message that the authors try to convey is lost.

Questions for authors

see above

Reviewer_oBnKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether a peer-conditioned visual world model can help multi-agent navigation. Each agent broadcasts its pose and intended future pose. The world model uses this information to generate future egocentric images, which are then used for collision avoidance. The paper finds that the broadcast improves image prediction but does not improve closed-loop planning. A state head predicts the neighbour’s position more accurately, but it learns the behaviour of the training population and fails when the neighbour follows a different policy. The paper therefore argues that agents should share poses and intents directly rather than communicate through generated pixels.

The paper has a high level of generative AI involvement. The authors state that AI was used for code development, experimental design and execution, statistical analysis, and manuscript writing. AI use is not itself a scientific problem, but the current paper needs stronger author-led organization and polishing. My score is 1: Clear Reject.

Critical strengths
  1. The interventional setup is useful. The matched zero-interaction control helps separate real agent interaction from simple visual correlation.
  2. The failure analysis is detailed. The oracle ladder separates errors caused by field of view, pixel readout, and world-model prediction.
  3. The state-head experiment gives an additional insight. Explicit state supervision improves position prediction, but the result still depends on the neighbour policy seen during training.
Critical weaknesses
  1. The declared-intent baseline is close to oracle information by design. The declared future is produced by rolling the neighbour’s own policy forward. This makes its strong performance expected, especially for the assertive neighbour. How does the method perform when the intent is noisy, stale, incomplete, or not followed?

  2. The conclusion is broader than the experiments support. The main evidence comes from one synthetic doorway environment with ray-cast images, coloured cylinders, a 90-degree field of view, and an NWM-style model. Please test a more realistic environment or limit the conclusion to this specific setting.

  3. The comparison mixes representation and policy information. The learned models are trained on yielding neighbours and tested on an assertive neighbour. The intent baseline directly receives the neighbour’s planned future. Would the result remain if the model were trained on a mixture of yielding and assertive policies?

  4. The presentation makes the paper difficult to follow. The figures and tables could be clearer, and the text is often dense. Question-style headings such as “Does It Help the Task?”, “Is It the Hand-Written Detector?”, and “Does the Conclusion Travel?” weaken the structure. Please use descriptive headings and organize the experimental settings, results, and interpretations more clearly.

Questions for authors

see weaknesses

Reviewer_8bbVOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies a shared world model in multi-agent navigation task. In a simulated navigation task, two robots must pass through a narrow doorway without colliding. The authors compare planning based on images generated by a world model with planning that directly uses the other robot’s broadcast information. They find that improving image prediction does not necessarily improve collision avoidance, and that directly using the other robot’s declared intent performs substantially better in this setting.

Critical strengths
  • The paper makes a useful comparison between two ways of using the same information. FULL and BROADCAST-INTENT both receive the other robot’s declared future pose. FULL uses this information to generate a future image and then estimates the robot’s position from that image. BROADCAST-INTENT uses the position directly. Comparing them tests whether generating an image actually helps the robot avoid collisions. The large performance gap provides clear evidence that this extra step can be harmful in the tested setting (Section 5, Table 2).
  • The experiments help explain where the image-based approach goes wrong. The authors first give the planner the other robot’s true future position, then replace this with information extracted from a true image, and finally from a generated image. These comparisons reveal errors caused by limited visibility, estimating positions from images, and generating the images themselves. This makes the negative result more informative than simply reporting that the world model performs poorly (Section 5, Tables 3–4).
Critical weaknesses
  • The main simulation setting is quite simple and toy-like, which limits how broadly the findings can be applied. The central task is to negotiate a narrow doorway in a hall with a dividing wall. This is useful for controlled experiments, but offers limited variation in obstacle layouts and environmental dynamics beyond the robots themselves.
  • The claim that a six-bit broadcast can match ORACLE is not clearly supported by the reported planning results. Appendix C shows that reducing each of the six broadcast values to one bit preserves much of the improvement in image prediction. Table 2 separately reports a 2.2% collision rate for BROADCAST-INTENT, without clearly saying that this test used the six-bit version. These are different outcomes: retaining image-prediction quality does not show that the compressed information is sufficient for safe navigation. The authors should test the six-bit version in closed-loop planning or narrow the abstract's claim (Table 2; Appendices C–D).
Questions for authors
  • Would the main conclusion still hold in a more complex environment, such as one with multiple obstacles, changing visibility, or independently moving objects that do not broadcast their intent? Could you compare FULL, the state head, and BROADCAST-INTENT under these conditions while keeping available observations and communication access comparable? This would help establish whether the findings extend beyond doorway negotiation, where the neighbour's declared future pose provides much of the information needed for coordination.
  • Does the 2.2% collision rate in Table 2 use the six-bit compressed broadcast, or the original uncompressed pose values? If it uses the original values, could you test whether the six-bit version achieves similar collision avoidance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.