- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies how groups of LLMs agents reach agreements during collaborative debates. The authors apply the Friedkin-Johnsen model to track how individual agent opinions evolve round by round. By testing this approach across various models, datasets, and network structures, they find that the model successfully reconstructs belief trajectories. The analysis also shows that group consensus is typically driven by a small number of stubborn agents rather than simple majority voting.
Critical strengths
The application of Friedkin-Johnsen to multi-agent LLMs debates provides a framework to study individual agent dynamics, identifying distinct roles driven by stubbornness and persuasiveness that determine final group outcomes. The authors perform a comprehensive empirical evaluation across multiple models, datasets, topologies, and group sizes.
Critical weaknesses
As discussed in the paper, the framework cannot capture scenarios where agents introduce entirely new answers that were absent from everyone's initial beliefs. This limitation restricts the utility in open-ended tasks where creative solutions might emerge during the debate. Moreover, while the model effectively reconstructs observed belief trajectories, fitted influence weights do not formally establish the direct causal impact of specific text messages. Addressing how linguistic content maps directly to numerical parameters remains an area for deeper validation.
Questions for authors
How sensitive are the fitted stubbornness and persuasiveness parameters to variations in the number of deliberation rounds or the specific timing of belief reports? Could the framework be extended to handle open-ended tasks where novel answers are generated during the interaction process?
Thank you for the positive review. We are glad you find the framework useful and the evaluation comprehensive. We have highlighted changes in the pdf submission in blue.
New answers. The fact that multi-agent deliberation in most cases does not leave the initial answer space is a finding, rather than a weakness of our approach. We do not influence the agents in how they update their beliefs but rather fit the FJ model to what we observe. Therefore, the agents not creating novel answers during the deliberation is a weakness of the multi-agent system. If debates regularly produced new answers, FJ would fail to fit the observed dynamics. Instead, only a tiny amount of trajectories fit poorly, and most of those introduce a new answer. So in practice, LLM agents rarely work out something new during a debate. They mostly redistribute the views they started with. We consider this one of the main conclusions of the paper and a limitation of current multi-agent debate rather than of our model. Section 6.1 now states this more clearly.
Text and parameters. We agree that a fitted weight is not a formal proof of the causal effect of a specific message. However, fitting a model to observed dynamics and interpreting its parameters, once the fit is good and the parameters pass sanity checks, is the standard approach in the social sciences and in statistical physics. Friedkin and Johnsen (1990, 2011) estimate influence weights from observed opinion change in human groups, and recent work applies the same logic to LLM agents (Abedini et al., 2026; Yazici et al., 2026; Okawa, 2026; El et al., 2026). Our sanity checks are the interventions in Tables 2 and 6, where the parameters move as expected whenever we know the ground truth.
To connect the parameters more directly to the text, we added an LLM-as-a-judge experiment that checks whether the private belief vectors agree with what agents write (Section 6.5). A judge model reading only the messages, without answer letters or beliefs, recovers the reported top answer in almost all cases, and identifies the more confident of two agents in most pairs.
We also added an intervention that removes the most/least persuasive agent and reruns the debate, keeping the initial messages of the other agents constant (Section 6.4). Removing the most persuasive agent changes the final beliefs more than removing the least persuasive one, so the fitted weights track which links in the text debate actually carry influence. Mapping specific linguistic content to the parameters remains future work.
Number of rounds and timing. The paper covers 5 and 10 rounds, and the fit stays stable across rounds (Figure 3). To test sensitivity directly, we refitted each trajectory on the first k rounds only (see Figure 13 in the appendix). Beliefs are reported in the same response as the message after every round, so the timing is fixed by the protocol.
Open-ended tasks. FJ works in any convex belief space, not only on the simplex. For free-text answers, one could map answers to a belief space, for example by clustering them into semantic answer classes or by embedding them. FJ can then be fitted on these representations. We added this to the discussion.
Changes based on this review