- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies receiver-side information handling as an explicit design dimension in LLM-based multi-agent systems. Instead of treating an incoming message as something that should always be followed, the paper considers three alternative handling policies (accept, challenge, local amend). The authors evaluate these policies using Qwen3-Max and GPT-5.6-Luna across ALFWorld, AppWorld, ToolSandbox, and Collab-Overcooked, and complement the end-to-end evaluation with controlled ALFWorld handoff experiments. Based on these observations, the paper proposes an evidence-gated adaptive policy that selects among the policies according to receiver-visible conflict and local repairability. Across eight model-environment combinations, the adaptive policy matches or exceeds the best fixed/reference condition in 7.
Critical strengths
-
Good problem formulation. The paper identifies a relatively under-explored aspect of multi-agent system design: not only who communicates with whom, but also what authority the receiver should assign to an incoming message.
-
Comprehensive controlled-mechanism analysis. The controlled ALFWorld experiments manipulate evidence ownership and receiver visibility, showing, for example, that CHALLENGE is highly effective when repair information remains with the sender, whereas LOCAL AMEND becomes much more effective when the relevant evidence is moved into the receiver's history. The matched-revision control further suggests that the benefit of CHALLENGE is not simply caused by giving the sender an additional model call, but is related to the diagnostic content of the feedback. This provides a reasonably convincing mechanistic story for why different handling policies can be preferable under different information conditions.
Critical weaknesses
- Instability of prompt-based strategy selection and self-judgment.
The adaptive policy assumes that the receiver can determine whether there is a "concrete local conflict" and whether a "reliable in-scope repair" is available, and these judgments are made within the receiver's ordinary generation rather than by an independently calibrated module. In this case, the evidence-gated policy relies on internal judgments that may be difficult for an LLM itself to make reliably. This creates a potential error-propagation issue: if the receiver incorrectly believes that it has reliable repair evidence, LOCAL AMEND may be selected when CHALLENGE or ACCEPT would have been safer. Empirically, The controlled routing experiment shows 93.3% overall route agreement, but the errors are concentrated at the CHALLENGE boundary, with only 60% recall for expected CHALLENGE cases.
Therefore, using a separate or trained module may make the judment more reliable.
- The end-to-end empirical evaluation is relatively small and heterogeneous.
The per-environment sample sizes are only 50 tasks for ALFWorld and 30 tasks for each of AppWorld, ToolSandbox, and Collab-Overcooked. Moreover, the environments use substantially different task structures and evaluation metrics, with ToolSandbox using a similarity score rather than binary success. This makes it difficult to assess the statistical robustness and generality of the reported point-estimate improvements. For example, several adaptive improvements are only a few percentage points, and the main adaptive table explicitly reports point estimates without significance marks.
Questions for authors
-
The current formulation emphasizes concrete conflict and reliable local repair. A particularly interesting case is when both sender and receiver possess evidence supporting different interpretations of the state. How does the policy behave when the two agents have conflicting but individually plausible evidence?
-
The paper assumes all agents are "innocent" (trying to make the task work). However, it's interesting to consider the author's method in a controversial scenario: one, or multiple agents in the system have malicious intentions and wants to ruin the task. Can the author's method still work in such cases? If so, the method can be used to enhance the robustness of a multi-agent system.