acceptodds
Under review as a conference paper at ICLR 2027

The Makeup of an Influencer: How Confidence Shapes Multi-Agent Debate

Abstract

As LLM-based multi-agent systems are increasingly used for collective decision-making, who gains influence during deliberation is a central question to predict the outcome and ensure the emergence of a trustworthy consensus. Despite complex, non-linear interactions between agents, we find that the final vote is well captured by a mixture-of-experts model that linearly aggregates agents' initial beliefs. Remarkably, its router can be approximated by a graph neural network operating solely on initial beliefs. This representation enables us to characterize how influence is allocated during deliberation. We find that influence is shaped by agents' self-assessed confidence, confidence as perceived by others, and initial alignment with other agents' views. Our findings reveal a surprisingly simple structure underlying multi-agent deliberation, with implications for both the efficient design of multi-agent systems and their vulnerability to overconfident opinions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, MASs, agentic networks, influence dynamics

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_gfk9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies which agents gain influence in LLM-based multi-agent debate on multiple-choice QA. Building on prior work that fits Friedkin–Johnsen opinion dynamics to LLM deliberation, the authors observe that the FJ equilibrium is a convex combination of initial beliefs, and therefore cast the multi-agent system as a mixture of experts whose router depends on the initial beliefs. They give a short theoretical treatment of belief-only and confidence-only routing, then run experiments with GPT-5.4 Mini and Qwen2.5 on MMLU-Pro, BBQ, and CSQA on several topologies and with role, expert, and communication-style prompts. They report that final beliefs are predictable from initial beliefs.

Critical strengths
  1. Understanding who steers a multi-agent debate matters for both efficiency and robustness, and the overconfident but wrong agent failure mode is a concrete, practically relevant vulnerability.
  2. The causal interventions are a good addition to the correlational analysis, and the hub-vs-leaf and stubbornness results are convincing.
Critical weaknesses
  1. Novelty relative to Bause et al. (2026) and Abedini et al. (2026). Contribution 1 appears to be largely established there; the paper should state precisely what is new.
  2. The theory is thin and mostly restates standard facts. Theorem 3.5 essentially assumes its conclusion (confidence orders agents by competence). More importantly, none of the quantities introduced are estimated empirically, so Section 3 does not actually constrain or explain the experiments.
Questions for authors
  1. What exactly is new relative to Bause et al. (2026) and Abedini et al. (2026)?
  2. In Table 2, the balanced, confident, and elaborate interventions all have confidence intervals that include zero. Which intervention effects are statistically distinguishable from the neutral baseline, and for which model?
3 comments

Thank you for the review. We are glad you find the problem relevant and the hub and stubbornness interventions convincing. We have highlighted changes in the pdf submission in blue.

W1 and Q1. Novelty over Bause et al. (2026) and Abedini et al. (2026). Abedini et al. (2026) primarily study the security implications of stubborn agents, who can take over a network. While they propose to use the FJ model to capture deliberation, some of the used prompts heavily steer the agents towards behaving in a way that follows the FJ model. In our work, rather than steering the agents regarding their belief updates, we ask what makes an agent influential, when interactions are not heavily steered. This allows us to study multi-agent debate in a less constrained and more realistic setting. Bause et al. (2026) introduce the MoE perspective and a first analysis of routing. Our submission goes beyond this initial analysis in four ways: First, we show that the outcome can be predicted from initial beliefs alone by a permutation-equivariant GNN, which beats the uniform average in all 60 of 60 outer folds. Second, we add a sharper theory of confidence routing. Theorem 3.4 quantifies what is lost when routing on confidence only, and Theorem 3.5 shows that confidence only needs to rank agents correctly and does not need to be calibrated. Third, instead of remaining purely observational, we investigate how interventions on prompts and on the graph affect the emerging influence. Fourth, in our work we conduct systematic experiments, spanning different topologies and group sizes. We clarified this in the related work section.

W2. Theory. The main point of the theory is to derive and explain our main insight: Multi-agent systems can be described as an implicit routing that is largely confidence driven. Theorem 3.5 is a sufficient condition by design. It shows that confidence does not need to be calibrated. It is enough if more confident agents tend to be more competent, which is a much weaker and more realistic requirement. Proposition 3.6 quantifies what happens when the condition fails. The theory also already explains direct empirical results. Theorem 3.4 predicts that routing on confidence alone loses information, and confidence weighting is indeed weaker than the learned router in every setting (Figure 29).

To strengthen the connection between the theoretical and empirical findings, we added Table 3. Even when the most confident agent is wrong, it is still the most influential agent in 52% (GPT) and 46% (Qwen-72B) of the runs, against 20% by chance. This is exactly the failure mode of Proposition 3.6.

Q2. Which interventions are significant? Based on the 95% intervals in Tables 1,2,4 the clear effects are the stubborn agent in both models (GPT +0.112, Qwen-72B +0.191), the two stubborn agents (both for Qwen-72B, the first for GPT) and the hub position (+0.129). The confident style raises influence in both models and the balanced style lowers it in both, but only the Qwen-72B effects exclude zero, and the elaborate style shows no clear effect. We agree that the style effects are small and should be read as suggestive.

Thank you for the clarifications. I’m raising my rating from 2 to 3. The rebuttal clarifies the contribution relative to prior work and which intervention effects are statistically significant. I still have some concerns about how directly the theory connects to the experiments.

We thank the reviewer for considering our rebuttal and for increasing their score. We appreciate their acknowledgment of the clarifications provided.

Reviewer_JvVNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors, in this work, look at how a set of LLM agents come to a collective decision. They use the Friedkin-Johnson (FJ) model for the interactions of the LLM agents. The authors claim that the final vote is well captured by a mixture-of-experts (MoE) model that linearly aggregates agents' initial beliefs and that its router can be approximated by a GNN operating solely on initial belief data.

Critical strengths

As far as I can see, I see the following as the contributions main strengths:

  1. Using a well-known and well-studies opinion dynamics model from social sciences to look at how LLMs interact to come up with a collective decision in interesting. Leveraging existing studies and results on the FJ model might give better insight on how a collection of LLM agents behave.

  2. Showing that LLM interactions are adequately described by FJ dynamics, and that influence weights can be predicted from only the initial beliefs of the LLMs could be useful.

  3. The MoE interpretation provides insightful information for analyzing how the relative confidence and competence of agents affect rho outcome.

Critical weaknesses
  1. The theoretical novelty is quite limited. Convergence properties of the FJ model and, more relevant to the manuscript, the fact that equilibrium is mainly determined by linear combinations of initial opinions are well-established (unless I am mistaken FJ themselves mention this in their original paper which the authors cite). The authors' MoE interpretation seems to be a reformulation of these known results. The authors should elaborate on exactly what is new in their work.

  2. If the graph and FJ parameters are fixed, the routing weights can be computed analytically (see the Proskurnikov and Tempo (2017) paper that the authors cite). If they vary, why is a GNN necessary and why is it better than a simpler alternative?

  3. The assumptions in the FJ model may not fit well with LLM agent interactions. These may involve nonlinear belief updates, reasoning-based corrections, and changes to the interaction structure.

Questions for authors
  1. FJ theory already establishes that equilibrium opinions are linear combinations of initial opinions. So, what specific new mathematical insight does the MoE interpretation provide?

  2. If the influence weights and other parameters vary substantially during deliberation, would the MoE model still stand? What if the agents substantially revise their beliefs following new evidence or persuasive reasoning?

  3. There has been substantial work done on alternate models (e.g., models where beliefs are modeled as Dempster Shafer beliefs, nonlinear or bounded confidence models that attempt to capture nonlinear and social judgement theoretic concepts (see Wickramarathne, et al (2014) "Convergence Analysis of Iterated Belief Revision in Complex Fusion Environments," IEEE J of Selected Topics in Sig Proc 8(4)598; Dabarera, et al (2016) "Consensus in the Presence of Multiple Opinion Leaders: Effect of Bounded Confidence," IEEE Trans on Sig and Inf Processing Over Networks 2(3)336).

3 comments

Thank you for the positive review and for seeing the value of connecting LLM debate with the FJ literature. We have highlighted changes in the pdf submission in blue.

W1 and Q1. What is new in the theory? We agree that the convex combination itself is classical, and Proposition 3.1 credits Friedkin and Johnsen (1990) and Proskurnikov and Tempo (2017) for it. The MoE view is not our main claim. Our central question is what makes an agent influential. What is new is that the FJ parameters change with the input, so the system acts as a router and not as a fixed ensemble. Building on this, Theorems 3.3 to 3.6 characterize when routing based only on initial beliefs or only on confidence can work, and what misrouting costs. The key insight is that confidence does not require absolute probabilistic calibration. It is enough if more confident agents tend to be more competent, and overconfident wrong agents are the failure mode. We revised Section 3 to make this distinction explicit.

W2. Why a GNN? The analytic weights require the FJ parameters, and these are only known after the debate has happened and been fitted. The GNN predicts the outcome before any debate, from the initial beliefs alone. Since FJ describes the debates well, the outcome is a mixture of initial beliefs, so predicting it is the same as identifying the routing weights descriptively. We compared the GNN with simpler alternatives, namely the uniform average, confidence weighting, an MLP, k-NN, and two new shrinkage baselines. Shrinkage moves every agent part of the way toward the group average, and confidence shrinkage moves it toward the confidence-weighted average. The GNN outperforms the uniform average and both shrinkage baselines in all six settings (Appendix F).

W3 and Q2. Nonlinear updates and changing parameters. We agree that LLM agents could in principle update nonlinearly or change how much they weight each other during the debate. Our results suggest that this plays a small role. FJ with fixed parameters fits the observed trajectories well in all settings, and when it is fitted on the first rounds only, it also predicts the next round well (Appendix D). This would not be the case if the weights changed strongly during the debate. In addition, the prediction experiment does not assume FJ at all. The GNN is not restricted to mixtures of initial beliefs, so it could also pick up nonlinear updates.

Q3. Alternative models. Thank you for the pointers. With probability vectors as beliefs, as in our setting, bounded-confidence models also update by averaging current beliefs, so their outcome is still a mixture of initial beliefs, only with state-dependent weights. The MoE view therefore extends to them. Dempster-Shafer beliefs would allow agents to express ignorance over sets of answers, which is an interesting extension. Both lines of work, including Wickramarathne et al. and Dabarera et al., are now discussed in Section 2.

I have read the author rebuttal. I am mainly satisfied with the rebuttal except for their claim that "bounded-confidence models also update by averaging current beliefs, so their outcome is still a mixture of initial beliefs." In bounded confidence, the update occurs only when the incoming information meets a certain threshold. Hence, the update does not occur in a linear fashion.

I will retain my score.

Thank you very much for the feedback. We agree. What we meant is that the final belief can also be expressed as mixture of initial beliefs. The mixture weights, however, can depend in a nonlinear way on the beliefs, inputs, confidence, etc. We will clarify this point.

Reviewer_1avROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how individual agents gain influence during LLM-based multi-agent deliberation on multiple-choice tasks (MMLU-Pro, BBQ, and CommonsenseQA). Building on recent work modeling multi-agent belief updates via the linear Friedkin-Johnsen (FJ) social influence model (Abedini et al., 2026), the authors observe that equilibrium beliefs under FJ dynamics form a convex combination of agents' initial belief distributions. By fitting FJ parameters to individual question trajectories, the authors interpret multi-agent deliberation as an input-dependent Mixture-of-Experts (MoE) router over initial beliefs. They provide a theoretical analysis of hard routing based on initial beliefs and normalized-entropy confidence under Brier loss, and empirically evaluate how initial confidence, relative confidence, majority alignment, network topology, and prompted personas/communication styles drive an agent's fitted FJ influence.

Critical strengths
  1. The paper conducts a broad empirical exploration across three models (GPT-5.4 Mini, Qwen2.5-14B, Qwen2.5-72B), three benchmarks, multiple graph topologies, and diverse prompting regimes (roles, communication styles, domain experts). The finding that surface-level stylistic cues (e.g., "elaborate" vs. "balanced" styles, or "teacher/doctor" vs. "rookie/student" personas) and prompted stubbornness reliably shift an agent's effective weight in group deliberation is an important practical insight for practitioners designing multi-agent workflows.
  2. Viewing iterative free-form LLM debate through the lens of an implicit router operating on t=0t=0 initial beliefs is a clean conceptual framing. Showing that simple non-communicative aggregations of initial beliefs (Uniform Average and Confidence Weighting in Figure 3 / Figure 20) already capture a large fraction of the final post-deliberation belief state effectively challenges the computational necessity of multi-round natural language debate for multiple-choice tasks.
Critical weaknesses
  1. In Section 4.1 (lines 292–293), the authors state: "We fit the FJ parameters independently to each observed belief trajectory." However, the paper never specifies the objective function, constraints, or optimization algorithm used to fit (Γ(x),A(x),W(x))(\Gamma(x), A(x), W(x)). For nn agents over T=5T=5 rounds on a complete graph, the model has nn parameters in Γ\Gamma, nn in AA, and n(n−1)n(n-1) off-diagonal parameters in WW (totaling n2+nn^2 + n parameters, or n2n^2 free parameters after the nn row-stochasticity constraints ∑jwij=1\sum_j w_{ij} = 1, e.g., 25 free parameters for n=5n=5 or 400 for n=20n=20) fitted to only T=5T=5 transitions. Because beliefs quickly converge to consensus, fitting a linear transition operator to a single short trajectory is severely underdetermined:
    • Any trajectory where Agent jj's initial answer happens to match the final group consensus can be trivially fit by setting γj≈1\gamma_j \approx 1 (high stubbornness) and routing weights toward jj.
    • By Proposition 3.1, M=(I−H)−1ΓM = (I-H)^{-1}\Gamma where H=(I−Γ)(A+(I−A)W)H = (I-\Gamma)(A+(I-A)W). Because Γ=diag(γ1,…,γn)\Gamma = \mathrm{diag}(\gamma_1, \dots, \gamma_n) multiplies (I−H)−1(I-H)^{-1} on the right while (1−γj)(1-\gamma_j) scales row jj of HH, γj\gamma_j both reduces agent jj's susceptibility in HH and directly scales column jj of MM, giving πj=γj[η⊤(I−H)−1]j\pi_j = \gamma_j [\eta^\top (I - H)^{-1}]_j. Consequently, the headline finding in Figure 4c that "An agent's influence is related to its stubbornness γ\gamma" (r=0.97r = 0.97) is mathematically circular: regressing or correlating πj\pi_j with γj\gamma_j conflates an algebraic dependency of the fitted parameterization with an empirical causal mechanism.
  2. A central claim in the Abstract and Introduction (Contribution 1) is that "its router can be approximated by a graph neural network operating solely on initial beliefs" and "learned models provide a further improvement [over uniform averaging]" (line 320). However, inspecting Figure 3 (page 6) and Figure 20 (page 27) reveals that this claim is not supported by the data:
    • On Qwen2.5-72B across all three datasets (MMLU-Pro, BBQ, CSQA) and on GPT-5.4-mini for BBQ and CSQA, the learned models (GNN/GAT, MLP, k-NN) achieve virtually the exact same MSE as naive Uniform Average and 1-parameter Confidence Weighting (e.g., MSE≈0.004\text{MSE} \approx 0.004 on MMLU-Pro for Qwen2.5-72B across all methods except Persistence).
    • Only on a single setting (GPT-5.4-mini on MMLU-Pro) do GNN/GAT and k-NN show a modest gap over Uniform Average (≈0.008\approx 0.008 vs. 0.0120.012). Moreover, Appendix E reveals that the GNN experiment was only run on a fixed 5-agent complete graph in the neutral condition—where the agent communication graph is trivial (all-to-all) and the "GNN" is actually a bipartite set-equivariant network over (agent, option) pairs rather than a GNN operating over varying multi-agent communication topologies.
    • In Section 4.1 (line 291), the authors state that "agent ii reports a belief vector bi(t)∈Δd−1b_i(t) \in \Delta^{d-1} over the dd answer options. These vectors are used only for analysis and are never shown to other agents." Yet while line 1456 states the base prompt instructs agents to output a distribution, the actual Base System Prompt box in Appendix F (page 28, lines 1460–1468) omits any probability/distribution instruction. It is unclear whether bi(t)b_i(t) is elicited as verbalized probabilities in text/JSON or extracted from token log-probabilities. If verbalized in text, LLMs are notoriously poorly calibrated and quantized (e.g., outputting 0.8, 0.2, 0, 0), which heavily distorts normalized entropy Cj(S)C_j(S).
    • The pooled regression and classification models include both raw Confidence Cj(S)C_j(S) and Rel. Confidence Rj(S)=Cj(S)/C(n−1)(S)R_j(S) = C_j(S) / C_{(n-1)}(S) simultaneously. Because Rj(S)R_j(S) is simply Cj(S)C_j(S) normalized by the runner-up confidence in that group (and Rj(S)≥1R_j(S) \ge 1 directly encodes whether agent jj has the highest confidence in the group), including both strongly redundant features splits Gini importance in Random Forests and inflates variance in Logistic Regression coefficients.
Questions for authors
  1. How exactly are the per-question FJ parameters (Γ(x),A(x),W(x))(\Gamma(x), A(x), W(x)) estimated from a single 5-round trajectory B(0),…,B(5)B(0), \dots, B(5)? What is the loss function and regularization? Given that n2n^2 free parameters are fit to 5n5n vector observations (which rapidly collapse to consensus after round 1 or 2), how do you ensure the parameters—and thus the equilibrium weights πj=γj[η⊤(I−H)−1]j\pi_j = \gamma_j [\eta^\top (I-H)^{-1}]_j—are uniquely identifiable rather than an artifact of the optimizer? Why use fitted FJ equilibrium weights πj\pi_j as the ground-truth definition of "influence" rather than directly regressing the actual observed final belief B(T)B(T) on initial beliefs B(0)B(0)?
  2. How are the latent belief vectors bi(t)∈Δd−1b_i(t) \in \Delta^{d-1} extracted at each round? Are they verbalized numerical probabilities requested in the prompt (the exact formatting instructions are omitted from the Base System Prompt box in Appendix F), or are they derived from next-token logits over the option letters?
  3. In the Abstract and Section 4.1, you claim that a GNN approximates the deliberation router and improves upon uniform averaging. However, in Figure 20, GNN/GAT has virtually identical MSE to Uniform Average and Confidence Weighting across 5 of the 6 model-dataset pairs, and is never evaluated on non-complete graph topologies (star, circle, chain, wheel). Can you clarify whether the GNN actually learns non-trivial routing beyond confidence/uniform weighting, and how it performs when predicting across the different graph topologies studied in Section 4?
  4. Proposition 3.1 frames MAS deliberation as a soft convex ensemble (∑jπjsj\sum_j \pi_j s_j), where combining multiple beliefs reduces variance. Why do Theorems 3.3–3.6 restrict attention exclusively to hard routing (g(S)∈[n]g(S) \in [n]), which selects a single agent and discards the variance-reduction benefit of soft routing? Can Theorem 3.4 be extended to soft simplex routers π(C)∈Δn−1\pi(C) \in \Delta^{n-1}?
2 comments

Thank you for the very careful review. We are glad you find the routing view clean and the style and persona effects useful.

W1 and Q1. Fitting and identifiability. We added a description of the fitting in Appendix D. We minimize the KL divergence between observed and simulated beliefs under the FJ constraints (γ\gamma and α\alpha in [0, 1], W row-stochastic with zero diagonal) and report the fit quality in Table 5 and Figures 26, 27 (Appendix D). To address identifiability independently of the optimizer, we added an analysis in Appendix D. By expressing the FJ update as a linear model, we assess whether the observed trajectories contain sufficient information to distinguish its effective coefficients, accounting for the estimated noise level. 74.0% of agents satisfy this practical identifiability criterion. We characterize behavior associated with failed identifiability and show that identifiability failures are more common when neighbors have very similar belief trajectories or when an agent's beliefs barely change (Table 6). Figure 28 shows concrete examples of trajectories that contain insufficient information, for example, to uniquely identify gamma from alpha.

An agent's influence depends explicitly on its stubbornness, as we can explicitly derive from the FJ model, which is an advantage of its analytic tractability. We mention this now also in the caption of Figure 4c. Note that the equilibrium influence weights also depend on the network interactions through (I−H)−1(I-H)^{-1}. The extent to which stubbornness translates into influence is therefore also dependent on the trust weights and the stubbornness of other agents. Examining this relationship gives us thus relevant information about how influence is established.

In a FJ model, the final beliefs are mixtures of the initial beliefs, so predicting the outcome from initial beliefs, as our GNN does without any FJ parameters, is a descriptive way to identify the routing (now explained in Section 4.1). Where π\pi is not identifiable, for example when agents start with the same belief, the specific choice of weights does not change the outcome. .

W2 and Q3. What does the GNN show? The GNN does not estimate the routing weights directly. It predicts each agent's final belief from the initial beliefs alone, which under FJ means that the routing is predictable from them (now explained in Section 4.1). It is also not a network on the communication graph but a permutation-equivariant graph network on agents and answer options. We agree that the differences are small in absolute MSE, since most debates end close to the initial majority and every model has a small error. They are, however, consistent. The GNN outperforms the uniform average, confidence weighting and two new shrinkage baselines in all six settings (Appendix F, Figure 29). The GNN is not always the best learned model (e.g., k-NN is slightly better on CSQA with Qwen-72B), but all learned models improve on the uniform average, which supports that the outcome depends on the initial beliefs in a way that averaging does not capture.

Our main experiment uses only the complete graph on purpose, because there the graph is the same for every debate and the GNN can be compared fairly with baselines that ignore it (Appendix F). To answer your question on other topologies, we added Appendix H. It uses the MMLU-Pro runs with GPT-5.4 Mini on all five topologies and compares the same GNN with and without the communication graph. In the version with the graph, information is passed along the edges of the debate. Knowing the graph makes only a small difference. The GNNs remain the strongest models on every graph (Figure 30, Table 7). A likely reason is that with 5 agents and 5 rounds, information reaches every agent even in the chain.

W3 and Q2. Belief elicitation. The beliefs are verbalized probabilities that agents report in their response and that are never shown to others. Appendix G now also reports an example task prompt. We use verbalized beliefs because this is what an agent communicates when asked how certain it is and is therefore also likely to implicitly affect the messages. Coarse values affect the absolute entropy, but Theorem 3.5 only needs the relative order of confidence.

Redundant features. We agree that confidence and relative confidence are correlated, which splits their Gini importance. We include both on purpose, since the question is whether absolute or relative confidence matters. Together they are the most important features in Figure 6. We have added models with a reduced set of features in Figures 19, 20 and 23a. However, we would like to caution that the generalization performance is slightly reduced compared to our full fit in the main paper, suggesting that also the other features have some relevance. Still, these additional model fits confirm that confidence and alignment are strong predictors of influence and verify the robustness of this insight.

Q4. Soft routing. We focus on hard routing to understand when confidence identifies the best expert and what happens when it fails. However, our results also extend to soft routing. First, since confidence is derived from initial beliefs, a confidence-only soft router cannot outperform the best belief-based soft router (Theorem 3.4). Second, by convexity of Brier loss, mixing beliefs is never worse than randomly selecting an expert with the same weights. Thus, soft routing can reduce errors through averaging. In the setting of Proposition 3.6, the error bound still applies when we replace the probability of selecting a wrong expert with the total weight assigned to wrong experts. We added a remark explaining this after Proposition 3.6. Thank you for pointing this out. This makes our theoretical intuitive argument more general.

We have highlighted changes in the pdf submission in blue.

Reviewer_HVNBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how influence is distributed in multi-agent debate (MAD) on multiple-choice question answering. The authors fit Friedkin-Johnsen opinion dynamics to agents' belief trajectories and interpret the resulting influence weights (pi = eta^T (I - H)^(-1) Gamma) as an input-dependent mixture-of-experts router. Their theoretical analysis examines when confidence-based routing selects competent agents and the cost of misrouting. Experiments with three LLMs on MMLU-Pro, BBQ and CSQA use five debate rounds and three seeds per configuration. The authors report that initial beliefs predict final beliefs, that confidence and alignment predict fitted influence, and that stubbornness prompts and hub positions increase influence. They interpret these findings as evidence that relative confidence drives implicit routing, with implications for MAD's vulnerability to overconfident agents.

Critical strengths
  1. The stubbornness and hub interventions give convincing examples of how an agent can gain influence. Prompting one agent to be stubborn raises its influence by 0.112 +/- 0.050 for GPT-5.4 Mini and 0.191 +/- 0.048 for Qwen-72B (Table 2). With two stubborn agents, both gain influence while the other agents mostly lose (Table 3). A star hub gains 0.129 +/- 0.048 over the complete graph (Table 1). These are interventions rather than observational associations, and the stubbornness effect appears across model families. These experiments illustrate the vulnerability discussed in Section 5.

  2. The prediction evaluation is careful, and the theoretical condition for confidence-based routing is useful. Appendix E uses question-level nested 10-fold cross-validation, keeps seeds from the same question in one fold, retrains on the full training folds and reports bootstrap confidence intervals. Across three datasets and two models, learned predictors improve on Uniform Average, with gains that are small in absolute MSE (for example, on GPT/MMLU-Pro, GAT has about 0.008 MSE versus 0.011 for Uniform Average; Figs. 3 and 20). Theorem 3.5 gives a useful sufficient condition: a correct relative confidence ordering, with suitable tie-breaking, yields optimal confidence routing without absolute calibration.

Critical weaknesses
  1. The evidence does not yet establish an input-dependent router driven by relative confidence.

(1-a) Predicting final beliefs does not by itself test routing. The GAT produces agent-specific outputs unconstrained by the convex hull of initial beliefs, whereas Uniform Average and Confidence Weighting produce a shared vector (App. E). Since debates do not fully converge (Fig. 1b), simple shrinkage toward the group mean could explain part of the gain. The paper does not test such a baseline or a convex-combination GAT, nor compare its learned weights with pi. Confidence Weighting also appears close to Uniform Average in most panels of Figures 3 and 20. These results support predictability, but not yet the claimed routing mechanism.

(1-b) The fitted influence measure needs validation. The paper fits roughly 30 FJ parameter entries per five-agent trajectory from five rounds, but does not detail the fitting procedure or report fit quality and parameter stability. Variation across questions could partly reflect estimation noise; no within-question, across-seed decomposition is given. The correlation with fitted stubbornness (rho = 0.97; Fig. 4c) is partly expected because pi is constructed from Gamma. Figure 5b's 0.91 classification accuracy also needs comparison with the roughly 0.8 majority-class baseline. Competence correlates with influence and relative peer influence (rho = 0.57 and 0.69, respectively; Fig. 18), while confidence and competence have a positive but variable correlation (rho = +0.63 +/- 0.61; Fig. 4a). It remains unclear whether confidence predicts influence beyond competence and initial majority alignment. Task accuracy is not compared with voting baselines.

(1-c) The confidence interventions are less convincing than the stubbornness intervention. The confident style changes influence by +0.048 +/- 0.051 for GPT and +0.051 +/- 0.045 for Qwen; Qwen's confidence change is also small (+0.013 +/- 0.029; Table 3). The elaborate effect changes sign across models (+0.032 +/- 0.051 versus -0.018 +/- 0.042). Calling these effects clear seems premature without defining the uncertainty and reporting paired tests. Stubbornness targets resistance to updating, with gamma fitted afterward, so its robust effect does not establish an effect of perceived confidence.

  1. The contribution over established work appears limited. Choi et al. (NeurIPS 2025) show that majority voting over initial answers accounts for most gains attributed to MAD. Zhu et al. (ACL 2025) link conformity to uncertainty in one's own prediction; ConfMAD (Lin and Hooi, Findings of EMNLP 2025) studies expressed confidence; and Regan et al. (ALIFE 2024) show that agents with fixed answers can strongly affect outcomes when placed at hubs. The equilibrium convex-combination formula is also classical, as acknowledged in Proposition 3.1. The main additions are Theorems 3.3-3.5, the GNN predictor and the interventions. The theorems are elementary and benchmark against the best single expert rather than the best convex mixture. Given the unresolved routing interpretation and the limited intervention evidence, I find the general mechanistic contribution insufficiently established.
Questions for authors
  1. How does the GAT compare with a simple shrinkage baseline and a version constrained to combine initial beliefs? Do the latter's weights agree with the FJ estimate pi, and is Confidence Weighting significantly better than Uniform Average?

  2. How are the FJ parameters fitted and validated, and how much variation is within questions across seeds versus between questions? Do Tables 1-3 report changes in pi or in I_j = pi_j / max_i(pi_i)?

  3. What does "+/-" mean in Tables 1-3, and do the confidence/style effects survive paired tests with multiplicity adjustment? When the most confident agent is wrong or disagrees with the initial majority, how often does it still have the largest pi?

  4. (regarding suggestion/future work) Varying stated confidence while holding answers fixed would directly test perceived confidence, ideally including a confident but incorrect agent among correct peers and comparing task accuracy with voting baselines.

4 comments

Thank you for the detailed review. We are glad you find the interventions convincing and the prediction evaluation careful. We have highlighted changes in the pdf submission in blue.

1-a and Q1. Does prediction test routing? The GNN does not estimate the routing weights π. Correct. However, it solves the same task showing that the multi-agent debate outcome (i.e. the final beliefs) can be predicted from the agents' initial beliefs. The existence of an implicit router follows from the FJ model. Under FJ, which fits the trajectories well (Appendix D), every final belief is a mixture of the initial beliefs with weights π\pi. If π\pi depends only on the initial beliefs, predicting the outcome from them and recovering the routing are the same task, at least where the weights matter. Where the weights cannot be identified, for example when agents start with the same belief, they do not change the outcome either. We revised the abstract and Section 4.1 to state this clearly.

To test whether the gain is just shrinkage toward the group mean, we added two baselines (Appendix F). Shrinkage predicts that every agent moves the same fitted fraction of the way from its own initial belief toward the uniform average. This captures debates that only partly converge, without any routing. Confidence shrinkage moves agents toward the confidence-weighted average instead, which combines partial convergence with routing by confidence alone. Both predict mixtures of the initial beliefs, as FJ does. Shrinkage hardly improves on the uniform average, and the GAT outperforms both shrinkage models in all six settings, so it captures routing beyond partial convergence and confidence. Comparing learned mixture weights directly with π\pi would require a GNN restricted to mixtures, which we leave for future work.

1-b and Q2. Fitting and stability. Appendix D now describes the fitting and reports the fit quality for all settings. It also checks for each agent whether its parameters are identifiable. Most agents pass this check. The others mostly have neighbors with nearly the same beliefs or barely change their belief, so their parameters hardly affect the fit. You are right that πj\pi_j depends explicitly on γj\gamma_j, so part of the relationship in Figure 4c follows from the FJ model structure, as we mention now also in the caption. It is actually an advantage of the analytic tractability of the FJ model that we can understand the role of γj\gamma_j explicitly. Note that the equilibrium influence weights also depend on the network interactions through (I−H)−1(I-H)^{-1}. The extent to which stubbornness translates into influence is therefore also dependent on the trust weights and the stubbornness of other agents. Examining this relationship gives us thus relevant information about how influence is established. We have added regressions in Appendix B to also include competence. Confidence remains the strongest predictor of influence (which is also intuitive, as we argue, since competence is not known a priori). Regarding the classification accuracy in Figure 6b (formerly 5b), we agree that predicting the majority class already gives about 0.8, since only one of five agents is the most influential. The regression (cross-validated R2R^2 of 0.72) is therefore the more informative result, and it shows that influence is largely explained by the considered variables.

1-c and Q3. Interventions. In Tables 1, 2 and 4, each value is the change in an agent's confidence or influence compared to the neutral debate, averaged over questions, with a 95% confidence interval. Influence here is the raw routing weight πj\pi_j. We have updated the definition in Section 4 to clearly separate πj\pi_j from the normalized influence IjI_j.

An effect is reliable if its interval excludes zero. This holds for the stubborn agent in both models and for the hub position. The confident style raises influence and the balanced style lowers it in both models, but these effects are small and only exclude zero for Qwen-72B. The elaborate style has no consistent effect, since it is positive for GPT and negative for Qwen-72B. We agree that the style effects are small and should be read as suggestive.

In the new Table 3, we report how often the most confident agent is the most influential agent, in general, and in cases where it is wrong or does not align with the initial majority. Even when it is wrong, it is still the most influential in about half of the runs (52% for GPT, 46% for Qwen-72B), against a baseline of 20% if confidence was unrelated to influence. When it is misaligned with the initial majority, it is the most influential only at chance level (GPT) or even less often (Qwen-72B). This is in line with our findings that next to confidence, alignment and, to a lesser extent, competence shape the influence of agents.

2. Novelty. Our contribution addresses a different question than whether aggregation helps. We study how influence is distributed across agents, and whether the influence weights are predictable. Choi et al. (2025) attribute much of the debate gains to majority voting, but this does not explain the non-uniform influence weights we identify. Zhu et al. (2025) and ConfMAD study the role of confidence in agent interactions. In our setting, confidence is never explicitly shown, yet it still drives influence. Regan et al. (2024) identify the importance of hubs in the network, our results go beyond this observation by studying the predictability of the emergent influence weights. We have added them to the related work section of our revised manuscript. Since the Brier loss is convex, mixing beliefs is never worse than picking one of them at random with the same weights. A soft router therefore has loss at most ∑jπjrj(S)\sum_j \pi_j r_j (S). Accordingly, our bounds also cover soft routing as an upper bound.

Q4. We would like to highlight that an agent’s stated confidence is not shown to the other agents. Therefore, only changing it, while leaving the message sent the same will not have an effect on the deliberation. However, indirectly changing the confidence (e.g. with prompts different from the communication styles) would be an interesting direction for future research.

Thank you for the rebuttal with detailed responses. Especially, the added shrinkage baselines and the analyses of FJ fit and identifiability addresses several previous concerns I raised, and I appreciate the authors for appropriately qualifying the small and inconsistent style effects. However, the GNN's predictions were not directly validated against the fitted FJ influence weights; and the proposed fixed-answer confidence intervention and comparison with voting baselines remain unaddressed. While I appreciate the authors' response during the rebuttal, I will retain my score.

Thank you for the follow-up. We would like to clarify what the GNN does and does not show.

Validating GNN predictions against FJ influence weights. The purpose of the GNN is to validate our insight that the multi-agent outcome can be predicted from the initial agent beliefs. We do not claim that it assigns influence to agents in the same way as the multi-agent system. It does not estimate the routing weights π\pi directly and we do not claim that any of its internal weights correspond to π\pi. In fact, we do not require the GNN to learn a router that linearly combines the initial beliefs. The link to routing comes from the FJ model, not from the GNN.

Accordingly, the GNN's attention weights cannot be compared directly with π\pi or with the FJ weights wijw_{ij}. First, the GNN works on a graph of agents and answer options, and agents are not connected to each other. There is therefore no attention weight between two agents. Second, wijw_{ij} describes how much agent i listens to agent j in each round of the debate, and it is fitted from the full belief trajectory. The GNN never sees these intermediate rounds. It maps the initial beliefs directly to the final beliefs with a nonlinear function. So its internal weights cannot represent a round-by-round interaction. Third, πj\pi_j combines an agent's stubbornness with its influence over all rounds, which again has no counterpart inside the GNN.

A comparison is possible in principle, for example by a form of linearization of the GNN, changing the initial belief of one agent and measuring how much the GNN's prediction changes. However, this analysis would depend on several choices. One has to decide whether the agent is removed or its belief is replaced and by what, how strong the change is, and how the effects are combined over agents and answer options to obtain a number comparable to π\pi.

We would like to distinguish this evaluation from the main main objective of our paper, i.e. to gain insights into LLM-based multi-agent systems and not GNNs that predict debate outcomes. The GNN serves to show that the final outcome is largely determined by the initial beliefs, confirming one of our insights. What makes an agent influential is studied directly on the fitted π\pi, through the analysis of confidence, alignment and competence and through the interventions.

Regarding the fixed-answer confidence intervention. We misread your suggestion in our first reply. To make sure we understand it correctly now, we take your proposal to be the following. One agent's chosen answer option is held fixed, and only the confidence expressed in its messages is varied.

Our confident-style intervention is intended to do this. Since the belief vectors are never shown to other agents, any effect of confidence on the other agents must be transmitted indirectly through the text style of the messages. The initial belief stays similar in the neutral run, as the confidence of the initial belief barely changes (Table 4). The prompted agent gains influence in both models in response to the intervention (excluding zero for Qwen-72B).

Regarding voting baselines. While our contribution is focused on how influence emerges during a debate and not what kind of criteria improve multi-agent performance, we have not compared accuracy. Nevertheless, to answer your question in the following, we compare the performance of a single agent, taking a majority vote over the initial answers, and the MASs after the debate and report the results in the following table:

DatasetModelRandomly chosen single agentInitial majority voteDebate (final majority)Debate − Initial majority vote [95% CI]
MMLU-ProGPT-5.4 Mini78.480.182.0+1.9 [0.4, 3.5]
MMLU-ProQwen2.5-14B54.156.859.6+2.8 [0.6, 5.1]
MMLU-ProQwen2.5-72B67.669.170.6+1.5 [0.2, 2.8]
BBQGPT-5.4 Mini90.391.091.4+0.4 [−0.3, 1.4]
BBQQwen2.5-14B85.786.587.4+0.9 [−0.1, 2.0]
BBQQwen2.5-72B87.988.487.5−0.9 [−2.0, 0.1]
CSQAGPT-5.4 Mini72.672.772.3−0.3 [−1.0, 0.0]
CSQAQwen2.5-14B77.377.378.3+1.0 [−0.7, 2.7]
CSQAQwen2.5-72B79.680.379.3−1.0 [−2.3, 0.0]

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment

We thank all reviewers for their constructive feedback that has strengthened our manuscript. We appreciate that Reviewer gfk9 increased their score to 3 and Reviewer JvVN indicated being largely satisfied with our rebuttal. Even though Reviewer 1avR did not answer our rebuttal and Reviewer HVNB did not acknowledge our explanation why GNN predictions and influence weights cannot be directly compared, we believe we have addressed their points of criticism.

1. Key improvements during rebuttal

  • FJ model validation (Appendix D): Added fitting details, goodness-of-fit and out-of-sample evaluations, and an identifiability analysis.
  • Stronger prediction evidence (Appendices F, H): Added two shrinkage baselines, both outperformed by our GNN in all six settings, and extended prediction experiments to five graph topologies.
  • Stronger evidence for confidence-driven influence (Appendix B, Table 3): Added controls for competence and correlated features. Confidence remains the strongest predictor. Even when the most confident agent is wrong, it becomes the most influential in 46–52% of debates (20% chance baseline).
  • Intervention analysis (Tables 1, 2, 4): Clarified statistical uncertainty, confirmed robust stubbornness and hub effects, and appropriately qualified weaker communication-style effects.
  • Theory, novelty, and baselines (Sections 2–3): Clarified contributions beyond prior work, extended the theoretical discussion to soft routing, expanded related work, and provided comparisons with initial majority voting.

2. Main contributions

  1. Predictability: We show that multi-agent debate outcomes can be predicted from initial agent beliefs alone, consistent with an implicit, input-dependent routing mechanism.
  2. Theory: We show that multi-agent debate can be cased as a mixture-of-experts with respect to the initial beliefs. We characterize when confidence-based routing succeeds, why relative confidence rather than absolute calibration matters, and analyze the cost of misrouting.
  3. Mechanistic insights: We identify confidence, relative confidence, and opinion alignment as central drivers of influence, including the vulnerability to confident but incorrect agents.
  4. Interventions: We analyze how stubbornness, network position, and communication behavior affect the distribution of influence.

Our central contribution is to explain how influence in multi-agent deliberation is largely driven by confidence and alignment with the majority, why this influence emerges, and when it becomes a vulnerability. We believe the revisions substantially strengthen the evidence for these findings and address the reviewers' concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.