- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies which agents gain influence in LLM-based multi-agent debate on multiple-choice QA. Building on prior work that fits Friedkin–Johnsen opinion dynamics to LLM deliberation, the authors observe that the FJ equilibrium is a convex combination of initial beliefs, and therefore cast the multi-agent system as a mixture of experts whose router depends on the initial beliefs. They give a short theoretical treatment of belief-only and confidence-only routing, then run experiments with GPT-5.4 Mini and Qwen2.5 on MMLU-Pro, BBQ, and CSQA on several topologies and with role, expert, and communication-style prompts. They report that final beliefs are predictable from initial beliefs.
Critical strengths
- Understanding who steers a multi-agent debate matters for both efficiency and robustness, and the overconfident but wrong agent failure mode is a concrete, practically relevant vulnerability.
- The causal interventions are a good addition to the correlational analysis, and the hub-vs-leaf and stubbornness results are convincing.
Critical weaknesses
- Novelty relative to Bause et al. (2026) and Abedini et al. (2026). Contribution 1 appears to be largely established there; the paper should state precisely what is new.
- The theory is thin and mostly restates standard facts. Theorem 3.5 essentially assumes its conclusion (confidence orders agents by competence). More importantly, none of the quantities introduced are estimated empirically, so Section 3 does not actually constrain or explain the experiments.
Questions for authors
- What exactly is new relative to Bause et al. (2026) and Abedini et al. (2026)?
- In Table 2, the balanced, confident, and elaborate interventions all have confidence intervals that include zero. Which intervention effects are statistically distinguishable from the neutral baseline, and for which model?
3 comments
Thank you for the clarifications. I’m raising my rating from 2 to 3. The rebuttal clarifies the contribution relative to prior work and which intervention effects are statistically significant. I still have some concerns about how directly the theory connects to the experiments.
We thank the reviewer for considering our rebuttal and for increasing their score. We appreciate their acknowledgment of the clarifications provided.
Thank you for the review. We are glad you find the problem relevant and the hub and stubbornness interventions convincing. We have highlighted changes in the pdf submission in blue.
W1 and Q1. Novelty over Bause et al. (2026) and Abedini et al. (2026). Abedini et al. (2026) primarily study the security implications of stubborn agents, who can take over a network. While they propose to use the FJ model to capture deliberation, some of the used prompts heavily steer the agents towards behaving in a way that follows the FJ model. In our work, rather than steering the agents regarding their belief updates, we ask what makes an agent influential, when interactions are not heavily steered. This allows us to study multi-agent debate in a less constrained and more realistic setting. Bause et al. (2026) introduce the MoE perspective and a first analysis of routing. Our submission goes beyond this initial analysis in four ways: First, we show that the outcome can be predicted from initial beliefs alone by a permutation-equivariant GNN, which beats the uniform average in all 60 of 60 outer folds. Second, we add a sharper theory of confidence routing. Theorem 3.4 quantifies what is lost when routing on confidence only, and Theorem 3.5 shows that confidence only needs to rank agents correctly and does not need to be calibrated. Third, instead of remaining purely observational, we investigate how interventions on prompts and on the graph affect the emerging influence. Fourth, in our work we conduct systematic experiments, spanning different topologies and group sizes. We clarified this in the related work section.
W2. Theory. The main point of the theory is to derive and explain our main insight: Multi-agent systems can be described as an implicit routing that is largely confidence driven. Theorem 3.5 is a sufficient condition by design. It shows that confidence does not need to be calibrated. It is enough if more confident agents tend to be more competent, which is a much weaker and more realistic requirement. Proposition 3.6 quantifies what happens when the condition fails. The theory also already explains direct empirical results. Theorem 3.4 predicts that routing on confidence alone loses information, and confidence weighting is indeed weaker than the learned router in every setting (Figure 29).
To strengthen the connection between the theoretical and empirical findings, we added Table 3. Even when the most confident agent is wrong, it is still the most influential agent in 52% (GPT) and 46% (Qwen-72B) of the runs, against 20% by chance. This is exactly the failure mode of Proposition 3.6.
Q2. Which interventions are significant? Based on the 95% intervals in Tables 1,2,4 the clear effects are the stubborn agent in both models (GPT +0.112, Qwen-72B +0.191), the two stubborn agents (both for Qwen-72B, the first for GPT) and the hub position (+0.129). The confident style raises influence in both models and the balanced style lowers it in both, but only the Qwen-72B effects exclude zero, and the elaborate style shows no clear effect. We agree that the style effects are small and should be read as suggestive.