- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TodyComm learns how agents should communicate as their behavior changes during a task. A recurrent controller assigns each agent a score, which is used to select participants, construct communication graphs, and choose the final answer. It is trained with the final task reward and evaluated on six benchmarks with agents that become unreliable during interaction.
Critical strengths
-
Adapting communication to observed behavior is a sensible and useful design. The method does not need labels identifying malicious agents or examples of the correct communication graph. This makes the approach more appealing than one that depends on knowing the failure type beforehand.
-
The experiments examine several important aspects of the method. Tests across attack mechanisms, agent counts, tasks, and model backbones provide reasonably broad support within the studied setting. The comparison with a more flexible edge-scoring model also helps justify the simpler node-score design.
Critical weaknesses
-
The theory assumes much of what the learning method needs to achieve. It assumes that the learned node scores are accurate and leaves the loss from greedy graph construction as an uncontrolled term. It therefore offers a conditional explanation, but does not establish that RL learns a good graph or adapts successfully after behavior changes.
-
The specific contribution of communication learning remains partly unclear. Section 5.5 shows that replacing learned final-answer selection with majority voting causes a very large performance drop. The ablation is useful, but it also shows that the gains cannot be attributed to communication graphs alone. A comparison using the same final-answer selection rule would better establish the value of the central contribution.
Questions for authors
-
How much improvement remains when competing communication methods use the same final-answer selection rule and a comparable token budget?
-
Is there evidence that the node scores actually learned by RL satisfy the theory’s approximation assumptions closely enough to give a useful bound in the main experiments?
Hi everyone, please acknowledge the authors’ rebuttal with a brief reply, even if your assessment is unchanged (if you already did or are about to do so, please ignore this message).