acceptodds
Under review as a conference paper at ICLR 2027

Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication

Abstract

Rate-limited multi-agent teams raise three questions the emergent-communication literature has answered only empirically: what an optimal message should encode, what compression costs over a horizon, and when a learned protocol is unique enough to be read by a teammate. We answer them for rate-limited Dec-POMDPs, and then measure how far reinforcement learning falls short of the optimum the theory locates. Our theorems fix what is achievable independently of any learner, so a gap between an engineered sender and a learned one at the same bit budget is an optimization fact rather than an information-theoretic one. We instantiate this on three MuJoCo continuous-control arenas spanning zero, partial and rigid physical coupling, with every condition charged exactly 2 bits per decision by construction, and the discriminating regime is produced by closing a physical side channel within one arena, holding bodies, task and reward fixed. Communication value is governed by coupling: where the agents are rigidly coupled through a shared object, no channel beats silence (, , ), because proprioception already carries what a message would say; where they are uncoupled, every condition solves the task; and in the partially coupled regime the engineered -bit sender reaches an interquartile mean of while the learned -bit sender reaches and is indistinguishable from silence (, ). Because the two senders hold the same alphabet, bandwidth cannot explain the gap. Initializing a learned run from an engineered run's receiver localizes the failure: the same channel then reaches against cold-started (), so the failure is neither representational nor a matter of maintenance: what reinforcement learning cannot do here is discover the protocol in the first place. Cross-play shows the learned protocols are individually meaningful and mutually unintelligible: self-play collapses to across seeds, and the best alignment we can construct leaves at least of that gap standing. Every headline result is reported at seeds per arena, against seven published baselines reimplemented in our trunk at matched rate.

Primary area:
reinforcement learning
Keywords:
Zero-shot coordination, Emergent communication, Multi-agent reinforcement learning, Rate-limited communication, Dec-POMDP

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dhUfOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether failures of learned multi-agent communication arise from limited bandwidth or difficulty discovering useful protocols. It develops theoretical results on task-relevant encoding, compression loss, and protocol identifiability, then compares engineered and learned senders at the same two-bit budget across three robot environments with different physical coupling. Engineered communication substantially outperforms learned communication in the partially coupled setting, while rigidly coupled agents perform well without messages. Warm-start experiments suggest a protocol-learning bottleneck, and cross-play tests show poor compatibility between independently learned protocols.

Critical strengths

The paper addresses a meaningful research question and follows a clear line of reasoning. The experimental study is comprehensive, with comparisons that closely match the main claims and support the conclusions.

Critical weaknesses

The manuscript's presentation and formatting appear somewhat rushed and would benefit from more careful editing.

Questions for authors

This paper offers a valuable exploratory study of how communication emerges. Although the presentation feels somewhat rushed, the experimental design is carefully reasoned, and I consider the paper above the acceptance threshold. I would appreciate clarification on the following points:

  1. The engineered protocol fixes both the information to transmit and its encoding. Does the difficulty of learning communication mainly come from selecting useful information, or from the need for the sender and receiver to adapt to each other? Can the experiments distinguish these two sources of difficulty?
  2. Does the observed gap indicate a persistent protocol-discovery barrier, or simply slower learning under the chosen optimization settings? The paper reports a run that appeared unsuccessful early but reached high performance at around 13 million steps. Given this observation, how is the 15-million-step budget justified? Although this limitation is acknowledged, the evidence seems insufficient to attribute the gap to a general protocol-discovery bottleneck.
  3. What literature supports the statement that “emergent-communication literature has, almost entirely, studied only one end of that axis”? Does this claim concern the entire field or specifically continuous-control studies?
  4. The statement that “A literature clustered there is measuring the one setting whose answer is known in advance” seems particularly strong. What prior evidence or theoretical result supports it? Under what conditions would physical coupling make explicit communication unnecessary?
  5. Could the contribution list be reorganized? Details of the experimental setup and procedure could go in the introductory overview, while the contributions should focus more clearly on the new findings and insights.
  6. Could the discussion under “The nearest neighbor, and why it does not settle the question” include other closely related studies? A broader comparison would help clarify the paper's position in the literature.
  7. Could the authors provide training curves for the metrics reported in Figure 1 and the other main comparisons? These would help readers assess convergence and interpret the results under the fixed training budget.
1 comment

We sincerely thank the reviewer for the positive assessment and constructive feedback/questions. We address each below. New results are on the bi-manual carry task with the bar hidden from the receiver, with 2525 seeds unless stated, and differences are seed-paired with 95%95\% confidence intervals.

(Q1) Selecting Information vs. Sender–Receiver Co-Adaptation: The evidence is consistent with co-adaptation. In the task, the sender observes the target directly, so its main challenge is encoding the target in a message the receiver can interpret and we fixed one side at a time (Table 2, Sec. 5.3). With a scripted sender, the receiver learns to act on the message (0.9360.936). With the engineered run's receiver frozen and the sender's message network learned from a random initialization, the run reaches 0.7720.772. When both sides learn, as in the standard learned sender, success is 0.5620.562, compared with 0.4940.494 for silent (no messaging). As a check that freezing by itself is not what helps, a frozen random receiver with nothing transferred gives 0.2970.297. Each side therefore learns when its partner is fixed and competent, and the failure appears when both adapt at once. We describe this as consistent with a joint protocol-discovery bottleneck; the controls do not separate it from other effects of two agents learning simultaneously.

(Q2) Persistent Barrier vs. Slower Learning: We acknowledge this feedback and have trained both senders to both 3030M and 5050M steps (Fig. 2b, Sec. 5.2). The gap is +0.374±0.125+0.374 \pm 0.125 at 1515M steps (0.9360.936 against 0.5620.562), +0.328±0.152+0.328 \pm 0.152 at 3030M (0.9360.936 against 0.6080.608), and +0.422±0.152+0.422 \pm 0.152 at 5050M (0.9740.974 against 0.5520.552, 1515 seeds). The learned sender does not improve with budget on average. The reviewer's observation is partly correct, and we now say so in Sec. 5.2: with a 3030M budget, 44 of 2525 learned seeds reach 90%90\% of their final success only after 1515M steps, although this does not change the mean.

We also tested what does close the gap for learning (Sec. 5.4, App. L). With auxiliary signaling and listening losses and 44 bits instead of 22, the learned sender reaches 0.8910.891 against 0.9530.953 for the engineered 22-bit sender (+0.062±0.164+0.062 \pm 0.164, 1515 seeds, not significant; we do not claim equivalence). Either change alone leaves a gap. At 22 bits, the learned sender with auxiliary losses remains below the engineered one on 2525 fresh seeds in a pre-registered test (0.7900.790 against 0.9530.953, +0.163±0.104+0.163 \pm 0.104, p=0.004p = 0.004).

(Q3) "Studied only one end of that axis.": Sec. 1 now limits the statement to the continuous-control studies of learned communication that we found, most of which use multi-agent MuJoCo, where a single robot body is divided among agents. App. B adds a table that places representative prior settings (referential games, particle environments, gridworld task allocation, and multi-agent MuJoCo) by whether the agents share a physical object and whether the receiver can obtain the sender's private information without a message.

(Q4) "The one setting whose answer is known in advance": We agree with this, and have removed this phrase. The supported statement is conditional: when the receiver's observations already reveal what the sender knows, a message cannot raise the achievable return. The rigid task is one such case (+0.001±0.001+0.001 \pm 0.001 over no communication), and the new experiments show the effect: the engineered message adds +0.441+0.441 with the bar hidden and +0.136+0.136 with it visible.

(Q5) Contributions: Yes, from the reviewer's feedback we have reorganized the list. We moved the setup and procedure to the overview in Sec. 1 and to Sec. 3, and the list now states findings.

(Q6) Related Work: For the related-work discussion (Sec. 2), we now covers Eccles et al. (2019), Lowe et al. (2019) and Hu et al. (2020) in addition to Hill et al. (2025), with the difference from each stated.

(Q7) Training Curves: We also added curves for the three tasks and for the 5050M runs on the bimanual task, with per-seed traces, means and 95%95\% bands (Fig. 6, App. J).

We also revised the presentation and wording throughout. We thank the reviewer again and believe the new controls, revised wording, and the coupling sweep address the concerns raised, and we would appreciate the reviewer's consideration for an improved score.

Reviewer_LPhWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper, in my opinion, is an example of highly AI-generated content that seems to have obtained some results, but it is impossible to decipher from the way it is written. I believe it's intending to study the emergence behaviour in multi-agent systems in the presence of communication, and tries to identify when and why communication helps. However, to be very honest, I cannot really fully understand what the problem setting exactly is, and what the conclusions of the paper are. It's written in a completely cryptic Claude-style, without a clear problem formulation, scenario description, or the definition of core mathematical variables (at least not in the expected order).

Critical strengths

I cannot identify any particular strength given I cannot really understand what the problem the paper is dealing with. The paper lists is contributions with sentences such as:

"Communication value is ordered by coupling, and the rigid regime is a null."

What am I supposed to understand from this? what is the "communication value", it has not been defined. What is "coupling" in this context? what is the "rigid regime"? A few paragraphs above it says "The construction is maximally, rigidly coupled." without properly explaining what coupling is, and what "rigidly coupled" means. And, what does "rigid regime is a null" mean exactly?

The paper is full of similar sentences and statements. Despite multiple attempts, I cannot decipher what it is trying to convey exactly. I got an overall feeling of the problem it is studying and some of its claims, but I cannot be fully sure for the stated reasons, and cannot fully evaluate the novelty or the correctness.

Critical weaknesses

As I've already stated, the paper (in my opinion) is fully generated by AI. The authors acknowledge AI-usage for various aspects, but I cannot see what human contribution there has been as the paper is not comprehensible to a human at all (at least not to me).

Hence, the core critical weakness is that paper is impossible to understand. There is no clear problem formulation explaining what scenario is considered, what the core variables are, etc. Whatever novel and interesting content there is, it is buried under a cryptic AI-style language, which puts the reader off. I do not see much effort on the authors side to make the paper and its conclusions accessible to human readers.

As an example, in the Setting section, where one expects to see a clear description of the problem setting and the formulation of the core problem, there are statements such as "Under filter stability (Dobrushin coefficient \rho <1) the value lost to compression is \Theta(\epsilon H) compared to \Theta(\epsilon H^2) in worst case, with a matching bound, ... " It's not clear what filter this is referring to as it has not appeared in the text before. Why is the Dobrushin coefficient relevant here? \rho has never been defined. What does "value lost to compression" mean? Reader doesn't even know at this stage what information is being compressed. What are \epsilon and H? Neither of them has been defined yet.

I can go on like this for each sentence in the paper.

Questions for authors
  • Which particular sentences in the paper were human generated, if any?

  • Can you explain in brief human-understandable sentences what problem you're trying to solve? What the scenario is, and what the main conclusions are?

1 comment

We sincerely thank the reviewer for their time and feedback. We agree with your feedback that we use terms such as "coupling" and "communication value" before defining them, and have rewritten the Introduction, added a Problem Formulation section (Sec. 3), rephrased the contributions, and revised the wording throughout the paper to be more understandable. We answer the two questions below.

(Q2) High level explanation:

Problem: Two robots must complete a task through collaboration, where one (the sender) knows which of four targets is active, but the other (the receiver) does not, due to partial observability. The sender can transmit 22 bits per control step. We are understanding: when such a message is helpful, and whether reinforcement learning can learn to use it.

Scenario: We use three simulated tasks that differ in how much the receiver can already infer about the target through the shared physical system, which we call "physical coupling". In the uncoupled task a drone and a ground vehicle never touch. In the partially coupled task two arms carry a bar; the receiver can normally observe the bar and infer the target from it, and we can remove that observation so that only a message reveals the target. In the rigidly coupled task two humanoids hold one rigid object and can sense each other's motion directly. In each task we compare silence (no messaging), an engineered sender (a fixed rule that transmits the target index), and a learned sender trained with reinforcement learning, all at 22 bits. The communication value is the success rate with a message minus the success rate without one.

Conclusion: First, whether a message helps depends on the physical coupling of the robots. Under rigid coupling no message improves on the silent (no-communication) baseline, whereas with the bar hidden, the engineered message raises success from 0.4940.494 to 0.9360.936. Second, in that setting the learned sender reaches only 0.5620.562, which is not significantly above silence (no-communication), however, the engineered sender shows that 22 bits are sufficient. The gap is therefore not explained by channel capacity. Third, when either side is fixed (a scripted sender, or a pre-trained receiver) the other side learns, while training both together fails even with three times the training budget. This is consistent with the difficulty lying in the sender and receiver having to find a protocol jointly. A learned sender reaches the engineered level only with auxiliary losses and twice the message size. Fourth, pairs trained separately develop protocols that other pairs cannot use. The theorems describe what is achievable at a given rate independently of any learner, which is the reference against which the second conclusion is read.

(Q1) AI Usage: We used AI for assistance in writing mathematical proofs, such as in Appendix D, and for verifying sections such as problem formulation to match standard notations. We accept that the responsibility for clarity is ours, and hence, have worked to improve clarity in key sections of the paper.

(W) Changes in the revision. Sec. 1 now describes the three tasks and the three senders clearly, and defines communication value, before any use of these terms. We added Sec. 3 (Problem Formulation), which defines the model, the channel, the rate R=log⁡2∣M∣R = \log_2 |\mathcal{M}|, the sender conditions, success and communication value, and physical coupling, in that order and before any result. Sec. 4.1 introduces each theorem, defining symbols before using them (HH is the horizon, ϵ\epsilon the per-step distortion that compression introduces into the receiver's belief, and ρ\rho the Dobrushin coefficient of the receiver's belief update).

We thank the reviewer again. We believe the rephrasing/revised sections make the problem, formulation, and conclusion clear, and we would appreciate the reviewer's consideration for an improved score.

Reviewer_xMtgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper gave an interesting and a controlled study of why learned communication fails in marl. It studies how physical coupling (they first view it as a variable, which is interesting and inspiring) affects the value of communication in MARL, then it develops theoretical results for rate-limited Dec-POMDPs and compares engineered and learned communication protocols under identical bandwidth constraints. Their main finding is that communication is most valuable under partial physical coupling, where the learned protocols struggle even though when they have enough bandwidths. The evaluations in 3 MuJoCo envs suggest that protocol discovery is the main bottleneck, not the communication capacity,

Critical strengths

First, this paper presents an interesting perspective, it provides useful insights into how physical coupling affects communication needs in multi-agent systems. Second, this paper has well-controlled experimental comparisons. The engineered and learned protocols use the same 2-bit budget, making their performance gap particularly informative, then the experiments also include multiple communication baselines, 25 random seeds, and warm-start analysis to investigate the source of communication failure.

Critical weaknesses
  • The experiments mainly rely on PPO in the MuJoCo environments, making it unclear whether the current findings could be extended to other MARL algorithms or more complex coordination tasks.

  • I did not see the isolation study of the protocol discovery bottleneck, the warm-start experiments transfer not only the receiver but also motor policies and the critic, while effectively increasing the total training budget. So the observed improvement cannot be attributed solely to easier communication protocol discovery. Although the authors acknowledge these factors, additional controlled experiments are still needed to strengthen their central claim.

  • The paper stated that they focus on emergent communication, but it actually provides limited analysis of what communication strategies emerge and could be transffered across tasks.

Questions for authors
  1. Would the engineered vs learned communication gap remain under other MARL algorithms, such as QMIX or alternative actor-critic methods? (or is there any reasons for selecting the PPO-based training in the paper?)

  2. Can the authors isolate the warm-start effect by transferring only the receiver's communication-related parameters while other components matched?

  3. In the paper, the authors said that physical coupling determines communication value. Could the authors provide experiments varying coupling strength continuously within the same environment, rather than primarily comparing three coupling regimes? I think evaluating the open-channel condition would help strengthen this statement.

1 comment

We sincerely thank the reviewer for their time and feedback. We have addressed all 3 of your main weaknesses and questions mentioned. All new results are on the bimanual carry task with the bar hidden from the receiver, with 2525 seeds, and differences are seed-paired with 9595% confidence intervals.

(W1, Q1) Testing other MARL Algorithms: We have extended MAPPO (centralized critic) runs to 2525 seeds (Sec. 5.2, App. I). Under MAPPO the engineered sender reaches 0.9440.944 and the learned sender is 0.5700.570, a gap of +0.374±0.121+0.374 \pm 0.121, compared to the 0.9360.936 and 0.5620.562 under IPPO (+0.374±0.125+0.374 \pm 0.125). We chose PPO-based learners because the tasks have continuous action space, however QMIX requires discrete actions. Sec. 6 now states that the result is shown for on-policy actor-critic methods and we had added off-policy learners as future directions.

(W2, Q2) Isolating the Warm Start: We agree with your statement that the original warm start also transferred the motor policies and the critic and added training budget, and we added controls for each in Table 2, Sec. 5.3. The reference is the learned sender trained from scratch for 15M steps, which reaches 0.562; and initialized from an engineered run, it reaches 0.857 (+0.295±0.131+0.295 \pm 0.131). Initializing instead from a no-message run, which provides trained motor policies and the same extra budget, gives 0.5250.525 (−0.037±0.167-0.037 \pm 0.167). Training from scratch for 30M steps, which matches the budget alone, gives 0.6080.608 (+0.046±0.095+0.046 \pm 0.095). Neither reproduces the gain. When the engineered run's receiver is frozen and the sender's message network is learned from a random initialization, the run reaches 0.7720.772 (+0.211±0.149+0.211 \pm 0.149), whereas freezing a random receiver with nothing transferred gives 0.2970.297.

We also ran the transfer you mentioned, copying only the receiver's message-input weights. It reaches 0.6740.674 against 0.6000.600 for a matched control that copies nothing (+0.074±0.130+0.074 \pm 0.130, not significant), so we cannot attribute the gain to the communication weights alone. We state this in Sec. 5.3 and Sec. 6, and we now describe the evidence as consistent with a joint protocol-discovery bottleneck, since the controls do not separate it from other effects of two agents learning at once.

(Q3) Continuous Coupling and the Open Channel: Regarding this feedback, we added both (Fig. 2a, Sec. 5.1, App. K). When the receiver can see the bar, the team with no message improves from 0.4940.494 to 0.8110.811, and the gain of the engineered message over no message falls from +0.441±0.129+0.441 \pm 0.129 to +0.136±0.114+0.136 \pm 0.114. For the continuous version, we pass the bar's state to the receiver through additive noise at six levels within the same environment, from fully hidden to nearly clear, keeping the bodies, task and reward fixed (1515 seeds per level). The gain of the engineered message falls from +0.51+0.51 with the bar hidden to +0.14+0.14 at the clearest level, and the within-seed trend is negative (mean Spearman ρ=−0.38\rho = -0.38, p=0.012p = 0.012). The learned sender does not exceed the no-message baseline at any level.

(W3) Analysis of What Emerges: On the uncoupled task, a 2525-seed probe shows that the learned message encodes both the goal sector and the bearing the receiver must steer (App. G). Cross-play shows that the protocols are specific to each training run: success is 0.9800.980 in self-play and 0.1440.144 across seeds (Sec. 5.5).

Additional Result: We also tested how much help a learned sender needs to succeed (Sec. 5.4, App. L). With auxiliary signaling and listening losses and 44 bits instead of 22, it reaches 0.8910.891 against 0.9530.953 for the engineered 22-bit sender (+0.062±0.164+0.062 \pm 0.164, 1515 seeds, not significant; we do not claim equivalence). At 22 bits, the learned sender with auxiliary losses remains below the engineered one on 2525 fresh seeds in a pre-registered test (0.7900.790 against 0.9530.953, +0.163±0.104+0.163 \pm 0.104, p=0.004p = 0.004).

We thank the reviewer again. We believe the new controls and the coupling sweep address the concerns raised, and we would appreciate the reviewer's consideration for an improved score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.