acceptodds
Under review as a conference paper at ICLR 2027

Friedkin and Johnsen Unveil the Dynamics of Multi-Agent Debate

Abstract

As LLM-based multi-agent debate drives advances in agentic AI, a central question arises: How does collective intelligence emerge from interactions among agents? We introduce a framework to study the underlying dynamics through the Friedkin–Johnsen (FJ) model of opinion formation. Across diverse empirical settings, the FJ model accurately reconstructs the evolution of agents’ beliefs while providing interpretable measures of stubbornness and persuasiveness that characterize how agents engage in consensus formation. We systematically compare FJ with alternatives, including DeGroot dynamics and a statistical-physics-inspired model, and find advantages in trajectory reconstruction, consistency with interventions, and interpretability of communication patterns. Analyzing different communication topologies and group sizes, we find that stubborn agents become less prevalent in larger groups, while persuasive agents become more prominent. Our results establish a simple, interpretable framework for understanding how individual interactions shape LLM multi-agent collaboration.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, MASs, agentic networks, social dynamics, Friedkin-Johnsen

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_2JXnOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how groups of LLMs agents reach agreements during collaborative debates. The authors apply the Friedkin-Johnsen model to track how individual agent opinions evolve round by round. By testing this approach across various models, datasets, and network structures, they find that the model successfully reconstructs belief trajectories. The analysis also shows that group consensus is typically driven by a small number of stubborn agents rather than simple majority voting.

Critical strengths

The application of Friedkin-Johnsen to multi-agent LLMs debates provides a framework to study individual agent dynamics, identifying distinct roles driven by stubbornness and persuasiveness that determine final group outcomes. The authors perform a comprehensive empirical evaluation across multiple models, datasets, topologies, and group sizes.

Critical weaknesses

As discussed in the paper, the framework cannot capture scenarios where agents introduce entirely new answers that were absent from everyone's initial beliefs. This limitation restricts the utility in open-ended tasks where creative solutions might emerge during the debate. Moreover, while the model effectively reconstructs observed belief trajectories, fitted influence weights do not formally establish the direct causal impact of specific text messages. Addressing how linguistic content maps directly to numerical parameters remains an area for deeper validation.

Questions for authors

How sensitive are the fitted stubbornness and persuasiveness parameters to variations in the number of deliberation rounds or the specific timing of belief reports? Could the framework be extended to handle open-ended tasks where novel answers are generated during the interaction process?

1 comment

Thank you for the positive review. We are glad you find the framework useful and the evaluation comprehensive. We have highlighted changes in the pdf submission in blue.

New answers. The fact that multi-agent deliberation in most cases does not leave the initial answer space is a finding, rather than a weakness of our approach. We do not influence the agents in how they update their beliefs but rather fit the FJ model to what we observe. Therefore, the agents not creating novel answers during the deliberation is a weakness of the multi-agent system. If debates regularly produced new answers, FJ would fail to fit the observed dynamics. Instead, only a tiny amount of trajectories fit poorly, and most of those introduce a new answer. So in practice, LLM agents rarely work out something new during a debate. They mostly redistribute the views they started with. We consider this one of the main conclusions of the paper and a limitation of current multi-agent debate rather than of our model. Section 6.1 now states this more clearly.

Text and parameters. We agree that a fitted weight is not a formal proof of the causal effect of a specific message. However, fitting a model to observed dynamics and interpreting its parameters, once the fit is good and the parameters pass sanity checks, is the standard approach in the social sciences and in statistical physics. Friedkin and Johnsen (1990, 2011) estimate influence weights from observed opinion change in human groups, and recent work applies the same logic to LLM agents (Abedini et al., 2026; Yazici et al., 2026; Okawa, 2026; El et al., 2026). Our sanity checks are the interventions in Tables 2 and 6, where the parameters move as expected whenever we know the ground truth.

To connect the parameters more directly to the text, we added an LLM-as-a-judge experiment that checks whether the private belief vectors agree with what agents write (Section 6.5). A judge model reading only the messages, without answer letters or beliefs, recovers the reported top answer in almost all cases, and identifies the more confident of two agents in most pairs.

We also added an intervention that removes the most/least persuasive agent and reruns the debate, keeping the initial messages of the other agents constant (Section 6.4). Removing the most persuasive agent changes the final beliefs more than removing the least persuasive one, so the fitted weights track which links in the text debate actually carry influence. Mapping specific linguistic content to the parameters remains future work.

Number of rounds and timing. The paper covers 5 and 10 rounds, and the fit stays stable across rounds (Figure 3). To test sensitivity directly, we refitted each trajectory on the first k rounds only (see Figure 13 in the appendix). Beliefs are reported in the same response as the message after every round, so the timing is fixed by the protocol.

Open-ended tasks. FJ works in any convex belief space, not only on the simplex. For free-text answers, one could map answers to a belief space, for example by clustering them into semantic answer classes or by embedding them. FJ can then be fitted on these representations. We added this to the discussion.

Changes based on this review

  • New LLM-as-a-judge experiment comparing beliefs and text
  • New experiment removing the most vs. least persuasive agent after round 1 (Section 6.4)
  • Refit on the first k rounds in the appendix
  • Extension to open-ended tasks in the discussion
Reviewer_AUi7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes to use the Friedkin–Johnsen (FJ) opinion dynamics model as a diagnostic tool for LLM multi-agent debate. Agents debate multiple-choice questions over several rounds and privately report a belief vector after each round; an FJ model is then fitted to each trajectory. The authors show that FJ reconstructs trajectories better than DeGroot, Potts, and averaging baselines, use the fitted parameters to classify agents as stubborn, persuasive, bridging, or dominant, report that stubborn agents become less common as group size grows, and examine the effect of prompt and topology interventions on the fitted parameters.

Critical strengths
  1. The research question of whether debate aggregates independent judgments or follows a few agents is important and well motivated.
  2. The stubbornness/persuasiveness decomposition is simple and interpretable.
  3. The experimental setup is straightforward, with controlled variations in communication topology and agent prompts.
Critical weaknesses
  1. The interpretation of fitted influence weights needs stronger validation. Low reconstruction error does not establish that the inferred weights reliably identify who influences whom. Section 6.4 shows how fitted parameters change under interventions, but does not test whether larger fitted influence weights correspond to larger effects on recipients' beliefs. For example, holding a recipient's prior history and other incoming messages fixed, does removing a message from a high-weight sender change the recipient's next belief more than removing one from a low-weight sender? Such a test would directly assess the interpretation of the weights as interpersonal influence.

  2. The comparison does not isolate the contribution of attachment to initial beliefs. The DeGroot baseline averages only neighbors' beliefs, whereas FJ additionally includes both self-retention (α\alpha) and attachment to initial beliefs (γ\gamma). Thus, Figure 2 does not establish which of these additions explains the improvement. A direct ablation would set γ=0\gamma = 0 while retaining a fitted αi\alpha_i for each agent and the same unrestricted WW as the full FJ model. The existing shared-WW variant also sets γ=0\gamma = 0, but simultaneously restricts WW, so it does not isolate the contribution of γ\gamma. Fitting on early rounds and evaluating on later rounds would also test whether the improvement persists on observations not used for fitting.

  3. The empirical advance over prior FJ-based studies remains limited. The paper builds on the FJ formulation and experimental protocol of Abedini et al. (2026), making its empirical findings central to the additional contribution. One potentially valuable finding is the shift from stubbornness toward persuasion as group size increases. However, the group-size analysis considers only three sizes (5, 20, and 50 agents), with the 50-agent condition evaluated only for GPT. The number of seeds used for each condition should be clarified, and uncertainty estimates should be reported for Figure 6.

Questions for authors
  1. Could the authors report held-out prediction error for the models in Figure 2, for example, by fitting on rounds 1,…,k1,\ldots,k and predicting rounds k+1,…,Rk+1,\ldots,R? This would help assess whether the improvement over DeGroot and Potts generalizes beyond the fitting data, given the differences in parameter count.
  2. How sensitive are the role distributions in Figure 6 to the thresholds γ>0.8\gamma > 0.8 and ψ≥2/∣Ni∣\psi \geq 2/|N_i|? Results for alternative thresholds (e.g., γ>0.7\gamma > 0.7 or 0.90.9, and ψ≥1/∣Ni∣\psi \geq 1/|N_i| or 3/∣Ni∣3/|N_i|) would help assess the robustness of the observed group-size trend. Ideally, all experiments would be evaluated over at least five independent random seeds, with variability reported; additional seeds are particularly important for the 20- and 50-agent settings.
  3. Could the authors clarify the main novel contributions relative to prior work, particularly Abedini et al. (2026), given the overlap in the model, protocol, and prompts?
  4. Could the authors conduct targeted causal intervention experiments to test the proposed local mechanisms? For example, modifying or removing particular neighbors' messages while holding the rest of the interaction fixed, and comparing the resulting changes in agents' responses with the model's predictions, would help establish whether the inferred mechanisms capture causal effects beyond observational fit.
2 comments

Thank you for the detailed review. We are glad you find the question important and the setup straightforward. We have highlighted changes in the pdf submission in blue.

General comment. What influence means in our setting. We would like to briefly recall what "influence" refers to in our setting, since it matters for which interventions are informative. In our debates, agents have no identity beyond what they have said so far and, in non-complete graphs, their position. In the neutral setting, all agents are copies of the same model with the same prompt. An agent's influence is therefore not a fixed trait that it would keep if its message were replaced. It is a property of what this agent says in this debate, together with where it is located in the graph. The fitted weights summarize how strongly each agent's answers pull on its neighbors over the whole debate. We do not claim that a single weight equals the causal effect of one message. This is the standard approach in the social sciences and in statistical physics, where a model is fitted to observed dynamics and its parameters are interpreted once the fit is good and they pass sanity checks. Friedkin and Johnsen (1990, 2011) estimate influence weights from observed opinion change in human groups, and recent work applies the same logic to LLM agents (Abedini et al., 2026; Yazici et al., 2026; Okawa, 2026; El et al., 2026). It holds equally for every alternative models we compare against.

For this reason, the most natural causal interventions target the two things that can make an agent influential independently of a single message, namely its prompted behavior and its position in the graph. These are the interventions in Tables 2 and 6. Prompted stubbornness increases fitted γ\gamma (+0.61 for Qwen-72B, weaker for GPT), removing an edge lowers the persuasiveness of its two endpoints, and star hubs are much more persuasive than leaves.

W1 and Q4. Do larger weights mean larger effects? Following your suggestion, we added a direct intervention test (Section 6.4, Figure 6, and Appendix F). For each debate (MMLU-Pro-hard, Qwen-72B, 333 questions), we identify the most and the least persuasive agent from the FJ fit. We then regenerate the debate from Round 1 with that agent removed. As a control, we regenerate it with no agent removed, which measures the variation from regeneration alone. We compare the remaining agents' beliefs with their beliefs in the original debate. On questions where the initial answers are not unanimous (n = 266), removing the most persuasive agent changes the beliefs much more than removing the least persuasive one, and removing the least persuasive agent is barely distinguishable from the control. When the initial answers are unanimous, no removal has an effect, as expected. The fitted persuasiveness therefore identifies the agents that actually carry influence. Interventions on single messages at single rounds remain an interesting direction for future work.

W2 and Q1. Contribution of γ. We added FJ with γ\gamma = 0 and with α\alpha = 0, both with the full W, to Table 8. Full FJ fits best in all settings, indicating that both γ\gamma and α\alpha are needed. Without γ\gamma, all agents on a connected graph are eventually pulled to one common belief, and the initial views only matter through inertia. With γ\gamma, one agent's initial view can keep pulling the group, which is the stubbornness effect our paper studies. Regarding held-out rounds, our framework is intended to describe observed debates, not to forecast them (Section 4). Nevertheless, we also evaluate the predictive performance by fitting the models on the initial rounds only and reporting their errors on the remaining rounds in Appendix C. We highlight example trajectories where specifically α\alpha or γ\gamma, either, or neither parameter is needed for accurately fitting the trajectory.

W3 and Q3. Contribution over Abedini et al. (2026). The mentioned work focuses primarily on the security aspect. While they originally proposed to use the FJ model to capture deliberation, they neither compare it against alternative models that could also fit the dynamics, nor do they establish whether it provides the most appropriate model. Additionally, several of the used prompts explicitly steer the agents towards behaving in a way that follows the FJ model. Our work proposes to use the FJ model as a descriptive framework for multi-agent deliberation, while not steering the agents regarding their belief updates. We investigate emerging communication patterns, conduct interventions on the agents, and systematically evaluate the fit of the FJ model. This allows us to assess the applicability of the FJ model, as well as to establish it as a descriptive model for multi-agent debate, independent of behavioral steering.

Q2. Seeds and thresholds. We agree that additional seeds are particularly relevant for our findings regarding the larger agent group sizes. We therefore extended our experiments to three seeds for groups of 20 and 50 agents with Qwen-72B, and 20 agents with GPT-5.4 Mini. We also added three-seed evaluations for Qwen-72B with star and wheel topologies. We now report 95% confidence intervals wherever applicable.

The thresholds follow the natural scale of the parameters. γ\gamma > 0.8 means an agent puts most weight on its initial view, and 2/∣Ni∣2/|N_i| means twice the uniform share. Thank you for your suggestion, we agree that varying the threshold ensures robustness of our finding and we have now added results for other thresholds for γ\gamma and persuasion to Appendix E. We observe that the shift from stubborn (and persuasive) to persuasive is robust to the specific choice of threshold.

Changes based on this review

  • Both ablations added to Table 8
  • Held-out error experiment in the appendix
  • Experiment that removes a persuasive/not persuasive agent and reruns the debate
  • Results for other thresholds in the appendix
Reviewer_VJs4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an interpretable framework based on the Friedkin–Johnsen (FJ) model to investigate how LLM agents’ beliefs evolve during multi-agent debate.

Critical strengths
  • The measurement design is clean. Agents debate in natural language while privately reporting belief distributions that are never shown to peers, so belief trajectories are observed without becoming part of the debate (Section 1; Fig. 1).

  • Question-level FJ fits reconstruct belief trajectories across three datasets (MMLU-Pro, BBQ, CSQA), two models (GPT, Qwen-72B), group sizes up to 50 and several topologies. They are benchmarked against six alternative dynamics models with explicit parameter counts (Section 6.2; Fig. 2; Appendix F, Table 5).

  • Separating stubbornness (γ) from persuasiveness (ψ) yields interpretable agent roles, and the interventions provide a useful sanity check: prompting agents to be stubborn raises fitted stubbornness, and removing an edge lowers the endpoints' persuasiveness (Section 6.4; Appendix Table 3). Failure cases are reported transparently (Section 6.1).

Critical weaknesses
  • The model comparison does not isolate innate-belief anchoring (γ). DeGroot is fit without self-weights (w_ii = 0; Appendix F.1), whereas FJ adds both belief retention (α) and prior-belief pull (γ). The only γ = 0 variant, FJ with shared W, also replaces the pairwise W with a single influence score per agent. Even so, it outperforms DeGroot in every setting (e.g., 6.67 vs. 12.33 ×10⁻³ on the 5-agent complete graph). Much of FJ's gain over DeGroot may therefore not require γ, and the remaining FJ vs. shared-W gap confounds γ with full W (Section 6.2; Appendix F; Table 5). Because stubbornness (γ > 0.8) is the paper's central interpretive quantity, this gap matters.

  • The group-size conclusion is confounded by how roles are counted (Section 6.3; Figs. 5–6). The 2/|N_i| threshold is normalized per receiver, but an agent counts as "persuasive" (or a "bridge") if it exceeds it for any one receiver. Larger groups therefore offer more chances to qualify, and W is fit with more free parameters per trajectory. Conversely, most debates contain a single stubborn agent regardless of group size (Fig. 5), so the falling share of stubborn agents in Fig. 6 is largely a denominator effect. A null baseline (e.g., uniform or permuted W) or size-invariant role measures are needed.

Questions for authors
  • Can you add full-W FJ with γ = 0 to Table 5? It is equivalent to DeGroot with freely fitted self-weights (w̃_ii = α_i, w̃_ij = (1−α_i)w_ij) and would isolate anchoring. Shared W with free γ would complete the 2×2.

  • How is "new answer" operationalized in Section 6.1: as an option that was no agent's initial argmax, or as a belief outside the convex hull of the initial beliefs? FJ mixtures can produce a new consensus argmax (averaging (.5, .4, .1) and (.1, .4, .5) gives (.3, .4, .3)), so only the latter is inconsistent with FJ. Please report the 88% figure under the convex-hull criterion.

2 comments

Thank you for the very precise review. We are glad you find the measurement design clean and that you value the transparent reporting of failure cases. We have highlighted changes in the pdf submission in blue.

Isolating γ\gamma. We ran the ablation you suggested and added two variants with a full W to Table 8, FJ with γ\gamma = 0 (equivalent to DeGroot with freely fitted self-weights) and FJ with α\alpha = 0. Full FJ fits best in every setting.

We agree that part of the gain over DeGroot comes from self-weights. However, the results do not support that γ\gamma is redundant. Removing γ\gamma worsens the fit in all three settings. This is exactly what we expect if most debates contain one stubborn agent, as reported in Figure 5. γ\gamma is not needed for every agent, but where it is needed it matters.

More importantly, γ\gamma and α are not interchangeable. Without γ\gamma, initial beliefs survive only through inertia, and on a connected graph all agents are eventually pulled to one common belief. Only γ\gamma lets an agent keep pulling the group toward its initial view, which is the stubbornness effect the paper studies. The interventions support this reading. Prompting an agent to be stubborn increases its fitted γ\gamma significantly.

Why the full FJ model. We use FJ in its standard form on purpose. It is one of the most studied and widely accepted models of opinion formation in the social sciences (Friedkin & Johnsen, 1990, 2011), with well-understood equilibria and a clear meaning for every parameter. This lets us connect LLM debate to a large body of existing theory and empirical work. As Table 8 shows, several reduced variants (FJ with shared W, FJ with γ\gamma = 0, FJ with α\alpha = 0) also fit well if one only looks at the reconstruction error. We nevertheless keep the full model from the literature, because our goal is not the smallest possible error but an established, interpretable description in which stubbornness and susceptibility are separate quantities. The ablations show that this choice does not cost anything in fit, since full FJ is best in every setting.

Group size and role counting. We do not think the group-size result is a counting artifact. Stubbornness is defined independently of group size, so the fact that the number of stubborn agents stays around one regardless of group size is precisely the finding. Larger groups do not produce more stubborn agents, so their share drops and influence spreads through persuasive and bridging agents instead. Figure 5 reports the absolute number of stubborn agents per debate, making clear why their relative share decreases with group size.

Persuasiveness, however, should be defined relative to group size. In a larger group, an agent has more neighbors it can persuade, but conversely, each receiver also has more neighbors to distribute its influence weight across. We therefore normalize by the number of neighbors of each receiver. Since each receiver distributes a total influence weight of one, the uniform baseline is 1/∣Ni∣1/|N_i|. Our threshold of 2/∣Ni∣2/|N_i| identifies influence weights at least twice this baseline, regardless of group size. The suggested uniform W would classify no agents as persuasive. In contrast, a constant threshold becomes progressively stricter relative to uniform influence as group size grows, making it less suitable for comparing persuasive roles across group sizes.

We appreciate the concern about threshold sensitivity and have added experiments with different persuasion and stubbornness thresholds, including a constant threshold (Figures 18–21). The constant threshold does change the resulting role distributions, consistent with its different interpretation across group sizes. Stricter degree-normalized persuasion and absolute gamma thresholds naturally move more agents in the “neither” category, however, the main shift from stubborn to persuasive roles remains consistent across the thresholds we tested. So we conclude our finding is robust to the choice of threshold within the defined persuasion metric.

Q2. New answers. Your example is correct, a new consensus answer does not by itself contradict the FJ model. We therefore have extended Section 6.1 to investigate deviations from the convex hull and find that badly fitted debates lie much farther from it than well-fitted ones.

We want to stress that this restriction of FJ is a feature, not a weakness. FJ can only mix the initial beliefs, so it must fail whenever a debate leaves their convex hull. The fact that it fits 99% of the debates on the hard questions shows that debate is not arbitrary. Agents redistribute existing views rather than discover new ones. A more flexible model might fit these trajectories equally well, but it would not reveal this underlying constraint. This is exactly what makes the good fit informative. Sections 6.1 and 7 discuss this observation.

Changes based on this review

  • FJ with γ\gamma = 0 and FJ with α\alpha = 0 added to Table 8
  • Role counts under alternative thresholds added in Appendix E (Figures 18–21)
  • Distance to convex hull
  • Section 6.1 now reports the distance of each debate to the convex hull of the initial beliefs
Reviewer_z3NCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors study how agent beliefs evolve during multi-agent LLM debates using the Friedkin-Johnsen model of opinion formation. The authors fit FJ models to belief distributions reported by agents across deliberation rounds for multiple-choice questions. They show that FJ can reconstruct these trajectories with lower error than several alternative opinion dynamics models. Using the fitted parameters of the FJ model, the authors characterize the stubbornness and persuasiveness of agents, and study their roles in consensus formation. They find that stubborn agents often drive consensus, whereas larger groups have fewer agents that are both stubborn and persuasive, and more agents that are persuasive without being stubborn.

Critical strengths
  1. The paper touches upon an interesting and important problem of understanding how individual agents influence each other during multi-agent deliberation and goes beyond just looking at the final consensus to study how it emerges.

  2. The metrics introduced are quite nuanced. The distinction between stubbornness and persuasiveness offers an interesting perspective to separate two different behaviors.

Critical weaknesses
  1. My main concern is whether the fitted FJ parameters genuinely capture inter-agent influence in natural-language deliberations. The authors optimize γ\gamma, α\alpha, WW to reconstruct observed belief trajectories, but a low reconstruction error only shows that FJ can approximate belief evolution, not necessarily that the recovered parameters reflect the actual dynamics of persuasion or stubbornness (since this is the key phenomenon under investigation).

  2. The influence matrix is learned entirely from changes in reported belief vectors, without considering the natural-language interactions themselves. How do we know that a high influence weight from agent jj to agent ii actually reflects that jj's arguments persuaded ii, rather than simply being one of several parameter configurations that explain the same trajectory? The paper acknowledges this limitation but still makes strong claims about persuasion and consensus without sufficiently validating the behavioral meaning of the fitted parameters.

  3. The entire framework depends on belief vectors, but the paper only states that agents report these distributions without clearly explaining how they are elicited or extracted. Are these self-reported probabilities, and is any validation performed to assess their reliability?

Questions for authors
  1. If the probabilities are self-reported, how closely do they agree with logit-derived answer probabilities, where available? This would help establish whether the reported distributions are reliable proxies for agent beliefs.

  2. Could different combinations of γ\gamma, α\alpha, WW produce similarly low reconstruction errors but lead to different conclusions about which agents are stubborn or persuasive? How stable are the recovered parameters across different fits?

3 comments

Thank you for the careful review. We are glad you find the problem important and the split between stubbornness and persuasiveness useful. We have highlighted changes in the pdf submission in blue.

W3, Q1. How are belief vectors elicited, and are they reliable? Beliefs are self-reported. In every round, each agent writes its message and, in the same response, a probability for each answer option. Only the message is forwarded to neighbors. The probabilities are never shown to other agents. The revised Section 4 and Appendix J provide the exact output format and parsing.

To test whether the reported beliefs match what agents actually argue, we added an LLM-as-a-judge experiment. A separate judge model (Gemma-4-31B) reads only the reasoning texts of two agents, without their answer letters or beliefs. Whenever a text commits to an option, it is the option with the highest reported belief in practically all cases. From the wording alone, the judge also identifies the more confident of the two agents in 72% of pairs (random chance would be 50%), and more reliably the larger their difference in reported confidence is. So the belief trajectories seem to agree with the text debate. We have added this experiment to Section 6.5.

It is known that logit probabilities and self reported beliefs do not necessarily align well (Kumar et al., 2024). While our model could also be tested on the former, we have decided to focus on the latter, as we study agent behavior in the debate rather than the underlying LLMs token-generation process. Token logits operate on a different level of abstraction. They describe how likely the underlying model is to produce a particular next token, given one sampled reasoning chain, rather than the agent's uncertainty about the answer itself. We use verbalized probabilities as the agent's own statement of how certain it is, which is also what a user or another agent would act on. We acknowledge that this does not mean the reported beliefs are connected to the underlying logit probabilities and have added a comment in Section 6.5.

W1, W2 and Q2. Do the parameters capture influence, and are they stable? We agree that a low reconstruction error alone does not prove that the parameters reflect persuasion. The paper already goes beyond fit in three ways. 1. Prompting an agent to be stubborn increases its fitted γ. 2. Removing the edge between two agents lowers the persuasiveness of exactly these two agents for both models (Table 6). 3. The hub of a star graph receives far more persuasion weight than the leaves (Table 2). To conclude, the parameters move in the expected direction whenever we know the ground truth.

Additionally, in response to reviewer AUi7 and to further confirm that the FJ parameters capture influence, we have added an experiment where we measure the effect of removing a persuasive vs. non persuasive agent to Section 6.4. We observe that indeed removing a persuasive agent has a greater effect on the beliefs of the agents, compared to removing a less persuasive agent.

We agree that the FJ parameters are not always uniquely identifiable. Following your suggestion, we have added a detailed identifiability analysis in Appendix B. We derive the identifiability conditions for the FJ model and evaluate practical identifiability on our observed debate trajectories. In the MMLU-Pro hard setting, 96.2% of agents satisfy our practical identifiability criterion for the effective coefficients, and 85.6% additionally satisfy the conditions for recovering the original FJ parameters. We characterize the remaining cases, which are often associated with very similar neighbor trajectories or beliefs that barely change over time. We also provide examples illustrating when stubbornness (γ\gamma), retention (α\alpha), either, or neither is needed to explain the observed dynamics.

We observed stable parameter estimates across repeated optimizer runs. However, we deliberately base our identifiability analysis on the information contained in the observed trajectories rather than optimizer convergence, since repeated fits obtaining the same solution does not guarantee uniqueness of the solution. The linearity of the FJ model dynamics is in fact an advantage that allows us to analytically analyze identifiability.

Our conclusions rely on thresholded roles (γ>0.8,ψ≥2/∣Ni∣\gamma > 0.8, \psi \geq 2/|N_i|), aggregated over hundreds of debates and multiple seeds, rather than exact parameter values from individual fits.

Changes based on this review

  • Belief elicitation now explained in Section 4 and Appendix J
  • LLM-as-a-judge experiment comparing beliefs and text (Section 6.5)
  • New appendix on identifiability with a counterexample and a condition that can be checked on the data
  • Removal of agent experiment

Thank you for the detailed rebuttal and considering my suggestions. I believe this analysis will benefit the paper. You touch upon some important topics in this field that will contribute to discussions around agent swarm behaviors. I have updated my score.

We thank the reviewer for seeing value in our work and for raising their score. We believe that the reviewer’s suggestions helped improve our submission.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment

We thank all reviewers for their constructive feedback that has strengthened our manuscript. We appreciate that Reviewer z3NC increased their score to 3. Even though the other reviewers did not reply or acknowledge our rebuttal, we believe that we have addressed their concerns with our answers.

1. Key improvements during rebuttal

  • Strengthened the validation of our framework (Section 6.4, Appendix F): Added interventions testing the relationship between fitted influence parameters and actual changes in agents' beliefs, including removing the most versus least persuasive agent and examining the effects of prompting stubbornness.
  • Addressed questions about parameter identifiability (Appendix B): Added a detailed theoretical and empirical identifiability analysis, including conditions for recovering the FJ parameters and an evaluation on our observed trajectories.
  • Expanded the experimental evaluation (Appendix C, Appendix E, Appendix H.1): Added ablations of the FJ model, held-out-round evaluations, additional seeds, and sensitivity analyses for the communication pattern thresholds.
  • Improved the validation of reported beliefs (Section 6.5): Added an LLM-as-a-judge experiment comparing agents' written reasoning with their reported beliefs.
  • Clarified the interpretation and limitations of our findings (Section 6.1, Section 7): Expanded the discussion of the convex-hull constraint, the contribution of stubbornness, the meaning of influence, and the applicability of our framework to open-ended tasks.

2. Main contributions

  1. A question-level model of deliberation. We fit FJ separately to individual multi-agent debates and show that its constrained dynamics closely reconstruct agents’ belief trajectories across diverse settings. We compare its fit with alternative dynamics models.
  2. Distinct routes to influence. We distinguish an agent’s attachment to its initial belief from its influence on other agents, enabling us to characterize stubborn, persuasive, bridging, and dominant roles. In the larger groups we study, stubborn agents become less prevalent and persuasion becomes a more prominent route to influence.
  3. Interpretable effects of system changes. We examine how group size, topology, and interventions on prompts and communication links affect fitted behaviors. We also identify cases that the FJ model cannot readily capture, particularly deliberations that introduce a new answer.

Our central contribution is providing a framework based on the FJ model for measuring and interpreting the dynamics of LLM-based multi-agent deliberation. Fitting the FJ model to individual questions, we can construct interpretable measures of agent stubbornness, influence, and susceptibility. This allows us to characterize how influence is distributed over agents and how it changes in different settings (topology, group size, interventions). We believe the revisions substantially strengthen the evidence for these findings and address the reviewers' concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.