- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CoS-Web, a multi-agent reasoning framework designed to address false consensus. Its COS component examines agents’ judgments to identify unresolved reasoning issues. These issues are then used by Spider-Web Communication to select relevant agents for local discussion. After each communication round, a judge checks whether the issues have been resolved and decides whether further interaction is needed.
Critical strengths
The paper aims to address false consensus problem of multi-agent system, where multiple agents agree on an answer based on the same unsupported assumption.
The paper proposes COS component to identify unresolved reasoning issues. These issues are then used by Spider-Web Communication to select relevant agents for local discussion.
Critical weaknesses
-
It is not clear whether D1 and D2 use the same agent and context that generate original answer, or trigger new agent calls. If D1 and D2 use the same agent (as shown in line 210-211: “Direct Suspicion (D1) performs individual scrutiny of other agents’ publicly visible judgments.”), there is no independent source for correcting a shared false assumption. Then criticizing and exchanging the same internal knowledge may not reliably correct the shared errors. If D1 and D2 does not trigger new agent calls, the method seems to encourage reconsideration and selective multi-agent debate.
-
The function of D2 and how it can evaluate residual risk are unclear. The anticipated defense is not generated by the criticized agent, and D2 does not receive any additional evidence. It is unclear how D2 can improve the reliability of the identified suspicions. The ablation study also does not demonstrate the effectiveness of D2. I suggest adding a D1+D3 variant and comparing it with the full D1+D2+D3 to isolate the contribution of D2.
-
The experimental evaluation is incomplete. The paper only evaluates Qwen3-14B-AWQ. It is unclear whether the findings generalize to other models or model families. In addition, the paper does not report the number of agents used for the multi-agent baselines. For a fair comparison, these baselines should use the same number of reasoning agents as the proposed method.
Questions for authors
-
Can you clarify whether D1 and D2 are performed by the same agent and within the same context used to generate the initial answer, or by independently instantiated critic agents? If the same agents are reused, what mechanism enables CoS-Web to correct an erroneous assumption shared by all agents, and how does this differ from existing self-refinement and sparse multi-agent debate methods?
-
How does D2 improve the reliability of the identified? Can you add a D1+D3 ablation to isolate the contribution of D2?
-
Do the results generalize beyond Qwen3-14B-AWQ? Also, how many agents are used for each multi-agent baseline, and are they configured with the same number of reasoning agents as CoS-Web?