acceptodds
Under review as a conference paper at ICLR 2027

Revisiting White-Box Communication in Multi-Agent Reinforcement Learning

Abstract

Communication is essential for coordinating the behaviors of multiple agents. However, existing black-box communication methods often rely on complex optimization mechanisms and produce communication behaviors that are difficult to interpret. Recent advances in large language models (LLMs) substantially reduce the effort required to construct white-box communication protocols, raising a natural question: can white-box communication become a competitive alternative to learned communication? In this work, we systematically revisit white-box communication by studying its effectiveness, sources of effectiveness, capability boundaries, and potential value for learned communication. We first show that LLM-generated white-box protocols achieve performance competitive with learned communication methods across decisions about what, when, and whom to communicate. Further analysis identifies advances in foundation-model capabilities as a key source of this effectiveness. We then characterize the capability boundaries of white-box communication. While scaling the number of agents alone does not necessarily degrade performance, increasing agent heterogeneity and protocol complexity poses greater challenges; performance also declines as coordination-relevant semantics become less explicit, and feedback-driven self-refinement remains unstable. These findings reveal a natural complementarity between white-box and learned communication: white-box protocols provide explicit communication knowledge but are difficult to refine reliably, whereas learned policies can continue improving through RL. We therefore use white-box protocols as teachers for neural communication policies, improving learning efficiency while retaining the capacity for further optimization. Taken together, our study demonstrates that white-box communication deserves renewed attention as a viable direction for multi-agent communication research.

Primary area:
reinforcement learning
Keywords:
Multi-Agent Reinforcement Learning, Multi-Agent Communication

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FdwFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper re-visit the white-box multi-agent communication protocols by going through an LLM model first, answering few questions by evaluating the llm-assisted communication effects, then it concludes that using LLM as the supervisor to guide the white-box communication first then use the learned communication policies could improve the performance.

Critical strengths
  • The writing of this paper is clear, and it separate lots of sections to re-visit the white-box communication and guide readers to its targeting research problems. Easy to follow the authors' thoughts.

  • This paper could be a good position paper about how the LLM could help the white-box multi-agent communication research when hand-designed communication protocols are needed.

Critical weaknesses

First, this paper's central ideas are not new, for the multi-agent communication, the pre-defined protocol can be transferred since the rules are fixed. This is a well-known conclusion.

Second, the papers conclusion is using the LLM-assisted white-box communication to guide the learning of comm policies. I don't see enough novelty with this idea, since using LLM as high-level planner for later primitive learning is not new, this is a widely-used method. And this paper uses very long texts and some experiments to get to this idea and sometimes the improvements of using the white -box communication are little.

Third, the only thing newly defined by this paper is the WB-Direct and WB-Train in section 4, then the paper begins to do evaluations to get conclusions that has been concluded by this community.

Questions for authors
  • Is the communication protocol generated by the final combined method emergent? Or still require pre-defined rules? Can the learned communication protocols transferred over different tasks?

  • The topology communication network mentioned in line 410 might be more interesting to illustrate and could be expanded more, instead of using long texts to introduce what scenarios or multi-agent tasks requiring white-box communication.

7 comments

W1: First, this paper's central ideas are not new, for the multi-agent communication, the pre-defined protocol can be transferred since the rules are fixed. This is a well-known conclusion.

Response. We thank the reviewer for the feedback. However, we believe that the assessment is based on a misunderstanding of our research motivation and contributions, which we would like to clarify below.

First, we respectfully disagree with the reviewer's statement that "for multi-agent communication, the pre-defined protocol can be transferred since the rules are fixed. This is a well-known conclusion." This assertion conflates the fixed nature of predefined rules with their cross-task transferability, which are fundamentally different properties. In fact, communication protocols are often closely coupled with task-specific observation semantics, agent configurations, and coordination requirements. For example, a white-box protocol designed for the SMAC scenario 1o_2r_vs_4r cannot be assumed to transfer effectively to 5z_vs_1ul, where different unit types and coordination requirements may necessitate substantially different communication decisions. Therefore, the fact that communication rules are fixed does not inherently guarantee their cross-task transferability, nor is such transferability a well-established conclusion in MARL.

Furthermore, as explicitly discussed in Lines 48–71 of the original paper, our motivation for systematically revisiting white-box communication stems from two key observations: (1) white-box communication offers advantages in interpretability and controllability over mainstream black-box communication approaches; and (2) recent advances in LLM capabilities have substantially reduced the manual effort required to construct and refine explicit communication protocols. Our motivation is not based on any presumed superiority of white-box communication in cross-task transferability. Neither white-box nor black-box communication policies can be assumed to transfer directly across different tasks without additional adaptation or task-specific considerations. Cross-task transferability constitutes a separate research problem and falls outside the scope of this study.

More importantly, the central contribution of our work is a comprehensive study that systematically revisits white-box communication from four perspectives: its effectiveness, sources of effectiveness, capability boundaries, and potential value for learned communication policies. These research questions are distinct from cross-task protocol transfer and collectively provide a broader understanding of the capabilities and limitations of white-box communication.

Therefore, we believe the reviewer's conclusion that "this paper's central ideas are not new," based on the presumed transferability of fixed communication protocols, is not well justified, as it does not accurately reflect the motivation, research objectives, or contributions of our study. Finally, given the reviewer's characterization of protocol transferability as a "well-known conclusion," we would appreciate it if the reviewer could provide relevant references or evidence supporting the claim that predefined communication protocols are inherently transferable across tasks simply because their rules are fixed.

W2: Second, the papers conclusion is using the LLM-assisted white-box communication to guide the learning of comm policies. I don't see enough novelty with this idea, since using LLM as high-level planner for later primitive learning is not new, this is a widely-used method. And this paper uses very long texts and some experiments to get to this idea and sometimes the improvements of using the white -box communication are little.

Response. We thank the reviewer for the feedback. However, we respectfully disagree with the reviewer's characterization that "the paper's conclusion is using the LLM-assisted white-box communication to guide the learning of comm policies," as it does not accurately reflect the scope and contributions of our work.

First, we would like to clarify the positioning and contributions of our study. As explicitly stated in Lines 17–19 and 80–94 of the original paper, our work is positioned as a comprehensive study rather than a method-oriented paper. With recent advances in LLMs substantially reducing the cost of constructing explicit communication protocols, our objective is to systematically revisit white-box communication by investigating four research questions:

(1) How Effective Is White-Box Communication?

(2) What Makes LLM-Generated White-Box Communication Effective?

(3) Where Are the Capability Boundaries of White-Box Communication?

(4) Can White-Box Knowledge Guide the Learning of Communication Policies?

Importantly, the first three parts are independent research investigations, rather than preliminary experiments designed to motivate the final framework of LLM-guided communication learning. These investigations provide several important findings, including the effectiveness of white-box communication under appropriate training settings, the influence of LLM capabilities on protocol quality, and the limitations of feedback-driven protocol refinement. Using white-box communication knowledge to guide neural communication learning constitutes only the fourth part of our study, rather than its central contribution. Therefore, the reviewer's characterization that "this paper uses very long texts and some experiments to get to this idea" overlooks the independent research objectives and findings of the first three parts.

Second, we respectfully disagree with the reviewer's novelty assessment based on the argument that "using LLM as high-level planner for later primitive learning is not new." LLM-assisted RL encompasses a broad range of research directions, including high-level planning, reward design, policy guidance, and communication. These directions address distinct research problems, and the existence of prior work on LLM-assisted planning does not, by itself, establish that the research questions investigated in our study have already been addressed. Specifically, our work investigates explicit communication protocols in MARL, systematically examining their effectiveness across what, when, and whom to communicate, the sources of their effectiveness, their capability boundaries, and their potential value for learned communication. While previous studies have explored LLM-generated communication mechanisms, our contribution lies in this comprehensive investigation rather than merely applying LLMs to generate communication rules or guide policy learning.

Third, regarding the reviewer's observation that performance improvements are sometimes limited, we emphasize that our study does not aim to demonstrate that white-box communication universally outperforms learned black-box communication. Rather, an important finding is that white-box protocols can achieve comparable coordination performance while retaining explicit and interpretable communication decisions. In particular, our additional WB-Train (Own Obs) experiments further demonstrate competitive performance under the same observation constraints as the black-box counterparts. Moreover, the additional ablation experiments in Section 7 demonstrate that both white-box supervision signals contribute to neural communication learning. These findings support the effectiveness and potential value of white-box communication, even when the performance improvements are modest.

In summary, our central contribution is a systematic investigation of the effectiveness, sources of effectiveness, capability boundaries, and potential value of white-box communication for learned communication policies. We therefore believe that the reviewer's conclusion that "I don't see enough novelty with this idea" is not well justified when the paper is evaluated primarily as a proposal for LLM-guided communication learning. We respectfully encourage the reviewer to reconsider the assessment in light of the paper's actual research objectives and findings.

W3:Third, the only thing newly defined by this paper is the WB-Direct and WB-Train in section 4, then the paper begins to do evaluations to get conclusions that has been concluded by this community.

Response. As clarified in our response to W2, the reviewer's statement that "the only thing newly defined by this paper is the WB-Direct and WB-Train in Section 4" does not accurately reflect the positioning and contributions of our work. As explicitly stated in Lines 17–19 and 80–94 of the original paper, our work is positioned as a comprehensive study rather than a method-oriented paper. WB-Direct and WB-Train are experimental settings designed to systematically investigate white-box communication, rather than the primary methodological contributions of our study.

More importantly, we respectfully disagree with the reviewer's characterization that our experiments merely reproduce conclusions already established in the community. Our study provides several distinct empirical findings:

(1) White-box communication effectiveness depends critically on its compatibility with MARL policy learning. Directly replacing learned communication with white-box protocols causes substantial performance degradation, whereas training MARL policies with fixed white-box protocols achieves performance comparable to learned black-box communication, even under the same observation constraints.

(2) The effectiveness of LLM-generated protocols depends on the underlying LLM capabilities and the specific communication problem. Stronger LLMs generally generate more effective protocols, while different communication dimensions exhibit different capability requirements, with what-to-communicate appearing more demanding than when or whom to communicate.

(3) The limitations of white-box communication are more closely associated with coordination complexity than team size alone. Increasing agent heterogeneity and reducing the explicitness of coordination-relevant semantics pose substantial challenges. Moreover, feedback-driven protocol refinement during MARL training remains unstable and does not reliably outperform fixed protocols.

(4) White-box communication knowledge can effectively guide neural communication learning. Despite the difficulties of directly evolving white-box protocols, their explicit communication decisions can serve as supervision for neural communication policies, improving learning efficiency while allowing further optimization through RL.

Collectively, these findings provide a systematic understanding of when and why white-box communication works, where its capability boundaries lie, and how its knowledge can benefit learned communication policies. Therefore, we believe that evaluating our study solely based on WB-Direct and WB-Train overlooks its broader research questions and empirical contributions.

Reviewer_iZdsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the effectiveness and limitations of LLM-generated white-box communication protocols in MARL. It represents decisions about whom to communicate with, when to communicate, and what to share as explicit rules, and evaluates performance under different schemes for replacing existing communication modules. The paper also examines the effects of LLM capability, team composition, coordination semantics, and feedback-based refinement. Finally, it presents an approach that uses white-box protocols to supervise neural communication networks.

Critical strengths

Broad analysis of the effectiveness and limitations of white-box communication. The paper examines several relevant factors, including LLM capability, team size, agent heterogeneity, coordination semantics, and feedback-based refinement. This analysis is valuable in identifying both the potential of white-box communication and the conditions under which its performance becomes limited.

Critical weaknesses
  1. Further methodological detail and experimental validation. Additional explanation and validation of key design choices and experimental procedures would help assess the proposed approach and its findings. For example, the design criteria for manually constructed protocols, the construction and use of refinement feedback, and the precise definitions of the supervision losses would benefit from clarification. Component-wise ablations and concrete examples of protocol generation and revision would also help establish the role and performance contribution of each component.

  2. Clearer support for the interpretability claims. The paper would benefit from clarifying what interpretability means within the proposed framework and which parts of the framework support this claim. For example, in Section 7, further optimization through RL may cause message selection and aggregation to differ from the teacher’s rules, leaving it unclear how the resulting message content remains interpretable. Concrete examples showing the meaning of individual message components and how they change during training would help substantiate this claim.

Questions for authors
  1. In the experiments in Figure 7, what environmental information is provided to the LLM, in what format, and how is it used to identify protocol weaknesses and guide revisions? Since the feedback design may influence refinement outcomes, could the authors provide an actual feedback example and clarify the criteria for accepting a revised protocol?

  2. Do the results reported over five random seeds include independent protocol generation for each run, or is the same protocol used across different MARL training seeds? Could the authors clarify whether variability across independently generated protocols was evaluated and how it was distinguished from variability due to MARL training?

6 comments

W1: Further methodological detail and experimental validation. Additional explanation and validation of key design choices and experimental procedures would help assess the proposed approach and its findings. For example, the design criteria for manually constructed protocols, the construction and use of refinement feedback, and the precise definitions of the supervision losses would benefit from clarification. Component-wise ablations and concrete examples of protocol generation and revision would also help establish the role and performance contribution of each component.

Response. We thank the reviewer for these constructive suggestions. To address these concerns, we have expanded the methodological details and experimental validation in the revised manuscript.

(1) Manual protocol construction (Appendix C). We clarify the design criteria for human-written protocols, including task relevance, observation grounding, and explicit communication behavior. We also specify how the LLM Coder translates human-defined rules into executable code without introducing additional strategies.

(2) Environmental feedback and protocol refinement (Appendix D). We detail the construction and use of structured environmental feedback, the Planner–Coder refinement process, and the acceptance and rollback criteria. We also provide a recorded IMMAC example tracing environmental feedback through failure diagnosis, rule revision, and evaluation.

(3) Supervision losses(Appendix E, lines1506). We provide precise formulations of LT\mathcal{L}_T and LM\mathcal{L}_M using weighted binary cross-entropy, clarifying their roles in supervising communication topology (when and whom) and message selection (what), respectively.

(4) Component-wise ablations (Section 7, Lines 519–525). We evaluate the removal of LT\mathcal{L}_T, LM\mathcal{L}_M, and both losses. As shown in Figure 9 in revised manuscript, removing either loss degrades performance, while removing both causes a substantial drop, demonstrating the contributions of both white-box supervision signals to neural communication learning.

Additionally, Appendix B provides a concrete example tracing protocol generation from task and observation descriptions to executable rules and validation.

Together, these additions directly address the requested methodological clarifications and provide further experimental evidence for the contributions of the proposed components.

W2: Clearer support for the interpretability claims. The paper would benefit from clarifying what interpretability means within the proposed framework and which parts of the framework support this claim. For example, in Section 7, further optimization through RL may cause message selection and aggregation to differ from the teacher’s rules, leaving it unclear how the resulting message content remains interpretable. Concrete examples showing the meaning of individual message components and how they change during training would help substantiate this claim.

Response. We have added a dedicated section in Appendix G to clarify the definition and scope of interpretability in our study. Specifically, Sections 4–6 investigate white-box communication protocols, whose communication decisions are explicitly determined by interpretable rules grounded in observation semantics. We provide a concrete example in Appendix B, illustrating how communication triggers, recipients, and message contents can be traced to specific rules and observation features. Section 7, however, addresses a different research question: whether explicit white-box communication knowledge can guide the learning of neural communication policies. Although the white-box teacher provides interpretable communication knowledge as supervision, the resulting student remains a learned black-box communication policy and may deviate from the teacher's rules through MARL optimization. Therefore, we do not claim that the final learned communication policy preserves the rule-level interpretability of its white-box teacher.

Q1: In the experiments in Figure 7, what environmental information is provided to the LLM, in what format, and how is it used to identify protocol weaknesses and guide revisions? Since the feedback design may influence refinement outcomes, could the authors provide an actual feedback example and clarify the criteria for accepting a revised protocol?

Response. We thank the reviewer for this constructive suggestion. To address these concerns, we have expanded Appendix D (Environmental Feedback and Protocol Refinement) to provide a detailed description of the feedback-driven protocol refinement process.

Specifically, (1) Environmental feedback: We clarify that the LLM receives structured JSON reports containing episode-level statistics (e.g., returns, win/loss outcomes, communication rates, and casualties) and selected timestep records (e.g., agent observations, actions, rewards, and communication decisions), together with the current communication protocol and observation metadata. (2) Protocol refinement: A Planner analyzes these records to identify potential communication weaknesses, such as missing communication, inappropriate recipients, or ineffective message content, and proposes corresponding revisions, which are then implemented by a Coder. (3) Acceptance criteria: Each candidate is evaluated over 100 episodes using the current policy parameters and is accepted only if its performance degradation and communication overhead remain within predefined tolerances. An additional rollback check is performed after 50,000 training steps.

Furthermore, we provide a concrete, recorded protocol refinement example using IMMAC on 1o_10b_vs_1r, illustrating the complete process from environmental feedback and communication weakness identification to rule revision, code implementation, and acceptance decisions. We also report protocol acceptance and rollback statistics, with 70 of 178 evaluated candidates accepted (39.3%).

These additional details clarify how environmental feedback guides protocol refinement and how candidate protocols are evaluated, thereby improving the transparency and reproducibility of our experiments.

Q2: Do the results reported over five random seeds include independent protocol generation for each run, or is the same protocol used across different MARL training seeds? Could the authors clarify whether variability across independently generated protocols was evaluated and how it was distinguished from variability due to MARL training?

Response. We thank the reviewer for this question. We clarify that all results in Figures 2, 3, 6, 7, and 8 are obtained using the same fixed white-box communication protocol across independent MARL training seeds. Therefore, the reported variability reflects randomness in MARL training rather than variability across independently generated communication protocols. We have explicitly clarified this experimental setting in the captions of the corresponding figures in the revised manuscript.

Variability across independently generated white-box protocols was not separately evaluated in this study. Our experiments aim to assess the effectiveness of a given white-box communication protocol under different MARL training seeds, rather than the consistency of protocol generation across independent LLM runs. We have clarified this distinction to avoid conflating protocol-generation variability with MARL training variability.

Dear Reviewer,

Thank you again for your time and constructive feedback. We would greatly appreciate it if you could take a moment to review our latest response and share any remaining concerns or comments. We would be happy to provide further clarification if needed. Thank you for your consideration.

Best regards,

The Authors

Reviewer_EXpyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper tests LLM-written communication rules in cooperative MARL. It replaces learned communication modules for what, when, or whom to contact, studies where fixed rules fail, and uses them to supervise a learned communication network.

Critical strengths

The automatic generation of communication protocols in multi-agent systems (e.g. in MARL) is a promising and valuable research direction.

Critical weaknesses

W1: LMAC already trains QMIX with a fixed LLM-written protocol and reports gains on overlapping SMAC maps. The novelty claim should focus on when/whom replacements, failure cases, and protocol-guided learning.

W2: The rules can read all agents' observations, while several replaced modules use local inputs. Without the rules or a local-only control, gains may reflect extra information.

W3: Protocol generation requires task and observation descriptions plus offline rollouts. The paper does not measure this setup effort or test transfer without regenerating the protocol.

W4: Claims about stronger LLMs and unstable refinement come from single-setting tests with underspecified protocol generation. LMAC reports different trends.

W5: The reported $0.20–0.30 generation cost lacks token counts, model prices, call counts, and repair costs.

Questions for authors

Q1: What additional information does an LLM need to develop a communication protocol, and how can a general MARL environment be adapted to such settings?

6 comments

W1: The novelty claim should focus on when/whom replacements, failure cases, and protocol-guided learning.

Response. We thank the reviewer for this suggestion. We have carefully reviewed the novelty claims in the Abstract, Introduction, and Related Work sections and confirmed that our central contribution lies in systematically revisiting white-box communication from four perspectives: its effectiveness, sources of effectiveness, capability boundaries, and potential value for learned communication policies.

Furthermore, we have revised the comparison with LMAC in the Related Work section to more explicitly clarify our contributions beyond prior work. While LMAC focuses on designing an LLM-generated communication-content selection mechanism for global-state reconstruction, our work provides a broader investigation across what, when, and whom to communicate. In particular, we examine the sources of effectiveness, capability boundaries, and failure cases of white-box communication, and further investigate how explicit white-box knowledge can guide the learning of neural communication policies.

These distinctions can be found in Lines 162–168 of the revised manuscript.

W2: The rules can read all agents' observations, while several replaced modules use local inputs. Without the rules or a local-only control, gains may reflect extra information.

Response. We thank the reviewer for this constructive suggestion. To further ensure a fair comparison, we introduce an additional setting, WB-Train (Own Obs), where each agent's white-box communication decisions are based solely on its own local observations, under the same observation constraints as the black-box counterparts. We evaluate this setting on three communication-focused SMAC tasks: 1o_2r_vs_4r, 1o_10b_vs_1r, and 5z_vs_1ul.

As shown Table below, WB-Train (Own Obs) achieves performance comparable to the learned black-box counterparts, although slightly below the original WB-Train setting. These results provide further evidence that white-box communication can achieve competitive coordination performance without relying on additional observation information.

TaskMethodOriginalWB-TrainWB-DirectWB-Train (Own Obs)
1o_2r_vs_4rIMMAC0.790.800.510.80
1o_2r_vs_4rSMS0.650.790.450.69
1o_2r_vs_4rM2I20.820.840.460.83
1o_10b_vs_1rIMMAC0.790.810.310.81
1o_10b_vs_1rSMS0.670.700.150.72
1o_10b_vs_1rM2I20.830.820.240.75
5z_vs_1ulIMMAC0.610.640.510.59
5z_vs_1ulSMS0.240.280.150.18
5z_vs_1ulM2I20.770.800.300.74

Detailed results and analysis are provided in Figure 2 and Lines 218–260 of the revised manuscript.

W3: Protocol generation requires task and observation descriptions plus offline rollouts. The paper does not measure this setup effort or test transfer without regenerating the protocol.

Response. We have clarified the setup effort required for white-box protocol generation in Appendix A.1 (Lines 803–816) of the revised manuscript. Specifically, (1) Task descriptions are concise, typically requiring only a single sentence describing the agent configuration, map, and task objective. (2) Observation descriptions are automatically extracted from environment, and the extraction code is reusable across different tasks within the same benchmark (e.g., different SMAC scenarios), without additional manual effort. (3) Offline rollouts require only 6,000 environment steps (approximately 60 trajectories), substantially fewer than the 2–5 million steps used to train black-box communication policies. Overall, these results suggest that the setup effort for white-box protocol generation is modest and manageable.

W4: Claims about stronger LLMs and unstable refinement come from single-setting tests with underspecified protocol generation. LMAC reports different trends.

Response. To draw more comprehensive and rigorous conclusions, we have expanded the evaluations in Sections 5.2 and 6.3 of the revised manuscript to cover all three communication dimensions: what to communicate, when to communicate, and whom to communicate with. Specifically, in Section 5.2 ("How Does LLM Capability Affect Protocol Effectiveness?"), we design a comprehensive evaluation setting using M2I2, SMS, and IMMAC on 1o_10b_vs_1r to systematically investigate the relationship between LLM capabilities and white-box communication effectiveness across these three dimensions. Similarly, in Section 6.3 ("Can White-Box Protocols Evolve Reliably?"), we extend the evaluation of feedback-driven protocol refinement to all three communication dimensions. The expanded experiments consistently support our original findings: (1) Stronger LLMs tend to generate higher-quality white-box communication protocols. (2) While white-box protocols provide effective and interpretable communication knowledge, their refinement through environmental feedback remains unstable and failing to consistently improve performance over fixed protocols.

Furthermore, we provide detailed descriptions of white-box protocol construction and refinement in the Appendix. Specifically, Appendix A presents the overall protocol construction framework, including the Planner–Coder workflow, offline rollout-based validation and refinement, and the setup effort required for protocol generation. Appendix B provides a detailed case study on 1o_2r_vs_4r, illustrating the complete prompt flow from task and observation descriptions to information-requirement analysis, communication rule design, executable code generation, validation, and optional offline refinement. Appendix D details the training-time protocol refinement procedure driven by environmental feedback, including the feedback format, Planner–Coder revision process, protocol acceptance and rollback criteria, and observed acceptance statistics. It also provides a concrete example illustrating how environmental feedback is translated into protocol revisions and subsequent evaluation decisions. Together, these detailed descriptions directly address the reviewer's concern regarding underspecified protocol generation by clarifying the generation workflow, prompt design, validation and refinement mechanisms, and implementation details. Overall, the expanded evaluations, together with the detailed descriptions of protocol generation and refinement provided in the Appendix, further strengthen the rigor and reliability of our findings.

Finally, we would like to clarify that our findings do not necessarily contradict those reported in LMAC, as the two studies investigate different protocol refinement objectives and settings. Specifically, LMAC focuses on improving global-state reconstruction through communication. Its refinement process employs an explicit state-awareness indicator to identify missing state information and inter-agent knowledge imbalance, providing targeted feedback for protocol refinement. In contrast, our study investigates whether LLMs can identify communication weaknesses from environmental feedback (e.g., failed trajectories and task returns) and autonomously revise communication protocols during MARL training. This setting poses additional challenges in identifying the causes of communication failures, as poor task performance does not directly reveal whether the problem arises from inappropriate communication decisions or the learned action policy. Moreover, LMAC performs offline protocol refinement before MARL training, whereas our experiments in Figure 7 investigate whether white-box communication protocols can evolve alongside MARL policies during training. Modifying communication protocols during training may introduce a mismatch with action policies that have adapted to previous communication behaviors, further complicating reliable refinement. Therefore, we believe these findings reflect differences in refinement objectives, feedback mechanisms, and evaluation settings, rather than contradictory conclusions about LLM-generated white-box communication.

W5: The reported $0.20–0.30 generation cost lacks token counts, model prices, call counts, and repair costs.

Response. We have added a detailed protocol-generation cost analysis in Appendix A.2 of the revised manuscript, covering API call counts, token consumption, model pricing, and code repair costs. Specifically, we report a representative case on 1o_10b_vs_1r, covering initial protocol generation, automatic code repair, and feedback-based refinement. The entire procedure involves 8 API calls, consuming 151,645 input tokens and 12,471 output tokens. Using unit prices of $0.00081 per 1,000 input tokens and $0.0132 per 1,000 output tokens, the total estimated cost is $0.28745, including $0.05200 for two code repair attempts. Since the original DeepSeek-V3 API was no longer available, we used DeepSeek-Chat with comparable pricing for this cost-accounting experiment. We also report the validation outcomes, including the unsuccessful rollout validation of the revised protocol. These additional details provide a transparent breakdown of protocol-generation costs and further substantiate our previously reported cost estimate of $0.20–0.30 per protocol.

Q1: What additional information does an LLM need to develop a communication protocol, and how can a general MARL environment be adapted to such settings?

Response. As detailed in Appendix A.1, our framework requires two main types of information for initial protocol generation: (1) task descriptions, specifying the agent configuration, map, and task objective, and (2) observation descriptions, specifying the semantic meaning of individual observation dimensions. The former can be provided by humans or LLMs, while the latter is automatically extracted from environment.

Regarding adaptation to general MARL environments, our framework primarily requires observation-description extraction and communication-interface adaptation. The observation extraction procedure is reusable across tasks within the same benchmark, while new environments may require corresponding extraction adapters. The generated white-box protocol then implements the communication functionality of the original module (what, when, or whom to communicate) while preserving its input/output interface. The validated protocol can subsequently replace the original communication module during MARL training.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

For clarity, we refer to Reviewers EXpy, iZds, and FdwF as R1, R2, and R3, respectively. We have provided detailed responses to each reviewer and revised the manuscript with additional experiments, methodological details, and clarifications. The changes in the revised manuscript are highlighted in blue for ease of reference. Below, we summarize the core contributions of our study and the principal updates made during the rebuttal period.

Core Contributions of Our Work

  1. Effectiveness across what, when, and whom to communicate. Directly replacing learned communication modules can substantially degrade performance, whereas training action policies with fixed white-box protocols can yield competitive coordination performance.
  2. Sources of effectiveness. In evaluated settings, stronger LLMs generally produce more effective protocols; message-content selection appears more demanding than communication timing or recipient selection.
  3. Capability boundaries. We examine agent heterogeneity, team size, coordination semantics, and feedback-driven refinement. Coordination complexity and less explicit task semantics pose challenges; protocol refinement during MARL training does not consistently improve upon fixed protocols.
  4. Value for learned communication. Explicit white-box decisions can supervise neural communication policies, improving learning efficiency while allowing further optimization through RL.

Additional Experimental Results During Rebuttal

  • Local-observation control — Figure 2 (R1; also relevant to R3). WB-Train (Own Obs) restricts each agent's white-box communication decisions to its own local observations. Tests cover IMMAC, SMS, and M2I2 on three communication-focused SMAC tasks, showing competitive performance under matched observation constraints, with task/method differences.
  • Expanded evaluation of LLM capabilities — Section 5.2 and Figure 4 (R1). We compared M2I2, SMS, and IMMAC on 1o_10b_vs_1r across what, when, and whom to communicate.
  • Expanded evaluation of protocol refinement — Section 6.3 and Figure 7 (R1). Experiments across all three communication dimensions show unstable refinement during MARL training without consistent gains over fixed protocols.
  • Ablation of supervision losses — Section 7 and Figure 9 (R2; also relevant to R3). Removing either the topology supervision loss or the message-selection supervision loss reduces performance in this setting; removing both causes a larger drop.

Additional Methodological Details and Clarifications

  • Relationship to LMAC — Section 2.3 and our responses to R1's W1/W4. We clarified our broader investigation of communication dimensions, effectiveness, capability boundaries, and protocol-guided learning, and how our training-time refinement differs from LMAC's offline refinement in objectives and feedback mechanisms.
  • Research positioning and scope — Our responses to R3's W1–W3 and Q1. Sections 4–6 provide independent findings alongside Section 7. Existing LLM-based high-level planning methods do not establish that these communication-specific questions have been addressed. Fixed rules do not guarantee cross-task transfer, which is outside our scope. In Section 7, the white-box teacher is required only during training; communication at execution is determined entirely by the learned neural policy. This is white-box-guided learned communication rather than fully emergent communication learned from scratch.
  • Protocol construction, setup requirements, and environment adaptation — Appendices A.1, A.3, B, and C (R1, R2). We detailed task and observation inputs, the Planner–Coder workflow, validation, manual protocol design, and environment-specific adaptation, with a case study from task descriptions to executable communication rules.
  • Protocol-generation cost analysis — Appendix A.2 (R1). We provided a breakdown of API calls, input/output tokens, model pricing, code repairs, and feedback-based refinement, with validation outcomes.
  • Environmental feedback and refinement procedures — Appendix D (R1, R2). We detailed structured feedback, revision, candidate acceptance criteria, and rollback checks, with a recorded refinement example and acceptance statistics.
  • Network architectures and supervision losses — Appendix E (R2, R3). We expanded TargetMatrix and MessageSelect specifications, including input/output dimensions and layer configurations, and defined the topology and message-selection supervision losses.
  • Interpretability and experimental variability — Appendix G and our responses to R2's W2/Q2. Rule-level interpretability applies to explicit white-box protocols; the neural student is not guaranteed to preserve it after RL optimization. We distinguished MARL training variability from protocol-generation variability, which was not separately evaluated.

As the discussion period approaches its end, please consider our responses and revised manuscript. We welcome remaining questions or concerns and would be happy to clarify them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.