acceptodds
Under review as a conference paper at ICLR 2027

Weakly-Supervised Cooperation in Decentralized LLM Multi-Agent Systems

Abstract

Existing state-of-the-art methods for learning inter-agent communication topologies in LLM multi-agent systems mostly learn through task outcomes. However, outcome-based rewards have significant challenges, such as requirement of correct task verification, a calibration phase, and also failing to adapt to changes in environment or team dynamics. In this work, we show that explicit task outcome signals are not necessary to learn effective communication topologies for cooperation. We propose a novel weakly-supervised method, MERIT-MAS, predicting dynamic and directed agent-to-agent communication topologies, only from agent-generated content. MERIT-MAS quantifies goal-conditioned relevance and novelty between agent pairs via cross-encoder rank scoring, and trains a topology network without using any task outcomes. Across established benchmarks and LLMs, our method achieves comparable performance (yielding overlapping 95% Wilson confidence intervals) with outcome-based SOTA baselines while spending 15% lesser agent-LLM tokens. Since MERIT-MAS does not need a calibration phase, it shows robustness under structural changes and scales seamlessly to larger agent counts, where methods that cache a single topology per episode collapse.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, llms, decentralized

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qRCPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

this paper tries to study such a problem: whether we can learn communication topology purely from agent-generated content, without explicitly conditioning on task outcome. The authors propose a method that uses goal-conditioned relevance and content novelty for weak supervision, instead of relying on task reward / outcome / calibration.

Critical strengths

the paper includes many good points such as well written, good presentation and relatively extensive experiments. But I find no substantive strength that can very strongly support a clear acceptance.

Critical weaknesses

there is a very critical concern before we try to study this problem: during the discussion stage, we actually can not reduce the risk of leakage of traditional task reward from the content of agents (because you can not change the generated content from agents but they may talk about such information), and I cautiously believe that the target problem is ill-defined.

the reported robustness/adaptability of MERIT-MAS seem to be generated by a fundamentally different training protocol from the baselines.

Questions for authors

None

1 comment

thanks for the response. I will maintain my score.

Reviewer_Mv5zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MERIT-MAS, which decides who reads whose messages each round based on the content published by the agents. The communication network does not use final task scores; it receives training feedback from message relevance and novelty and continues updating during operation. The authors evaluate performance using three LLMs in standard cooperation tasks, under team changes, and with team scaling, comparing success rates and token consumption.

Critical strengths
  1. During testing, the method's average success rate is close to that of G-Designer. Its average agent-LLM token consumption per task instance is 111k, compared with 132k for G-Designer.

  2. Learning communication relationships without relying on final task scores is an idea worth investigating, particularly in settings where outcome feedback is difficult to obtain. The experiments on the G-Designer network in Table 4 also support this training approach.

Critical weaknesses
  1. The results after team changes still lack controls that explain where the gains come from

    When relationships within the team change, the proposed method continues updating its parameters, while the comparison methods have already stopped updating theirs. It is therefore still unclear how much the new training signal contributes. If the proposed method also stopped updating before the change, how much worse would its subsequent performance be? What would the results be if training continued with the two signals using the same network, the same update frequency, and comparable budgets? Table 4 only covers standard tasks and does not answer these questions. In the scaling experiments, methods that are trained with task rewards and adjust the topology every round also continue to work. The current results are not sufficient to show how much of the adaptability emphasized in the paper comes from the new training signal.

  2. The scoring method cannot express different message rankings for different readers

    Different agents may be missing different information. For example, A needs C's messages but not D's, while the opposite is true for B. However, this choice cannot be represented by Equation (1) as written: the reader and source scores are directly added together. When comparing two sources, the reader-specific part does not affect the ranking, so all readers rank these two sources in the same order. The paper argues that the preceding self-attention can represent each reader's preferences, but it does not change the final additive scoring structure. The authors need to clarify whether the actual model uses Equation (1) and how this limitation affects message selection and task performance.

Questions for authors
  1. How much did continued learning contribute after the team changed? If updates to the network designed in this paper stopped before the change, how much would its subsequent performance differ from that with continued updates? Table 4's comparison of the two training signals only covers standard tasks. In tasks involving changes in team relationships, what would the results be with task rewards versus the feedback used in this paper, keeping the network and update frequency the same and the training budgets comparable?

  2. Regarding the scoring method in Section 3.2, under Equation (1), the ranking of two candidate sources shared by different readers in the same round does not change with the reader. Does the actual implementation also use this form? Different agents may need different messages. How does this limitation affect the fit to the relevance targets and the final performance?

Reviewer_fYZmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper shows that explicit task outcome signals are not necessary to learn effective communication topologies for cooperation. We propose a novel weakly-supervised method, MERIT-MAS, predicting dynamic and directed agent-to-agent communication topologies, only from agent-generated content.

Critical strengths
  • The studied problem is solid, interesting, and probably very important in the future, as we move into the agent era and agents are talking!
  • The challenges summarized from the existing work is clear and makes sense: outcome-based rewards have significant challenges, such as requirement of correct task verification, a calibration phase, and also failing to adapt to changes in environment or team dynamics.
Critical weaknesses
  • I think the writing can be improved. First, I think the problem is interesting and practical, but then when I try to grasp more technical insights, I am confused: Why does the paper raise a central question like "can an LLM-MAS communication topology be learned purely from the agent-generated content, without explicitly conditioning on task outcome?" I mean, why would learning topology be the central problem in multi-agent problems?
Questions for authors
  • Why are most baselines about training? The title is ``weakly cooperation in multi-agent llm systems'', which is super general, but why would the baselines be narrowed down to using rewards or some sort of signals to learn? Related to my first concern, why learning topology is the central problem?

Comments on the paper

1 comment

Hi everyone, please acknowledge the authors’ rebuttal with a brief reply, even if your assessment is unchanged (if you already did or are about to do so, please ignore this message).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.