acceptodds
Under review as a conference paper at ICLR 2027

Seeing the Forest for the Trees: Emergent Agreement from Individual Conversations

Abstract

As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi-turn conversations where two LLMs discuss subjective matters, can agree or disagree, and may change their stances. Using a linear classifier on individual conversation activations, we identify an internal representation of agreement. We find that it varies substantially across turns and conversations and does not reliably track external agreement at the individual conversation level. We therefore move beyond the representation of agreement in individual conversations, by aggregating the agreement representation across conversations. Externally, this aggregate view reveals a simple structure: when pooled across conversations, agreement can be predicted well with a Markov process, with agreement at each turn being a direct function of agreement at the previous turn. Similarly, when internal agreement is aggregated, it couples with the external Markov agreement through a power-law mapping, exposing Markovian dynamics in internal representations. Additionally, we find that after a change of stance, models express agreement externally before their internal representations reflect the same shift. These results reveal a joint structure between internal and external agreement that emerges only when internal features are aggregated across conversations.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Markov Chain, Hidden State, Consistency, Agent, Agreement, LLM conversation, LLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bno6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how agreement emerges in conversations between LLM agents. The authors analyze both model outputs and hidden representations. They show that behavioral agreement can be approximated by a first-order Markov model. They also find a relationship between aggregated internal agreement and behavioral agreement, and report that internal agreement lags behind behavioral changes.

Critical strengths

The paper addresses an interesting question about agreement in multi-agent systems. I particularly like the idea of studying both internal representations and observable behavior.

The experiments cover several models and datasets. The paper is also clearly written, and the results are easy to follow.

Critical weaknesses

Section 4 shows that a first-order Markov model fits the aggregated agreement trajectories reasonably well. However, this does not necessarily mean that the underlying process is Markovian. Earlier conversation history may still influence the next state.

The authors use a linear classifier to find an agreement direction, but I couldn't find how well it actually predicts agreement on the test data. I'm also not sure whether it captures agreement or just the model's chosen answer or persona.

I'm also not fully convinced by the power-law relationship. It uses three parameters and is fitted on very few data points. Some of the results in Table 2 are quite weak. I wonder whether a simpler function, like a linear one, would work just as well.

Questions for authors

Have you checked whether earlier turns also affect agreement, rather than just the previous one? I think this would help support the Markov assumption.

How well does the classifier actually predict agreement on test conversations? Could it be picking up the model's chosen answer or persona instead?

Have you tried fitting simpler functions, like a linear one, instead of the power law? I'm curious whether they would give similar results.

Could the internal lag simply come from the fact that the two groups have different conversation histories, rather than from a real delay in internal agreement?

4 comments

We thank the reviewer for appreciating that our paper studies both internal and behavioral data, the breadth of our experiments, and the clarity of the paper.

Below, we address your main concerns.


Section 4 shows that a first-order Markov model fits the aggregated agreement trajectories reasonably well. However, this does not necessarily mean that the underlying process is Markovian. Earlier conversation history may still influence the next state.

We agree with the reviewer; our paper does not claim that agreement is Markovian but that it can be approximated by a Markov model. To address the concern that earlier conversation history may still influence the next state, we further show in Section 4 that a second-order Markov model has only a small improvement to the approximation, indicating that a first-order Markov model is a good approximation of the behavioral agreement.

The authors use a linear classifier to find an agreement direction, but I couldn't find how well it actually predicts agreement on the test data. I'm also not sure whether it captures agreement or just the model's chosen answer or persona.

Thank you for this comment. We added an analysis of the internal agreement classifier AUROC and a visualization of the separation of agree and disagree labels in Appendix E. We show that the direction separates agreement from disagreement well above chance for all models: the AUROC is 0.74 for Ministral, 0.84 for Qwen, and 0.99 for Gemma. This analysis indicates the usability of our agreement direction. We further show that these AUROC results are consistent even when classifying between examples in which the speaker chose the same answer, holds the same persona, and comes from the same dataset. This indicates that the internal representation captures agreement beyond the model’s persona and chosen answer.

I'm also not fully convinced by the power-law relationship. It uses three parameters and is fitted on very few data points. Some of the results in Table 2 are quite weak. I wonder whether a simpler function, like a linear one, would work just as well.

We thank the reviewer for this comment. The power-law experiment aims not only to detect correlation but also to identify a specific function that captures it. Thus, given that correlation, regardless of form, can already indicate a possible relationship between the Markov and internal agreement, we added the Spearman correlation results at the start of Section 5.1. The results show a high correlation between the internal agreement and the Markov behavioral agreement (mean Spearman correlation across datasets = 0.85/0.95/0.93 for Ministral/Qwen/Gemma; p<0.05 in 14 of 15 model--dataset pairs). We also calculated Pearson correlation, which showed high correlation but lower than Spearman (mean Pearson correlation across datasets = 0.78/0.82/0.91 for Ministral/Qwen/Gemma; p<0.05 in 14 of 15 model--dataset pairs). Because Pearson is lower than Spearman, this suggests some non-linearity in the correlation. Note that in the power-law function, γ=1 is equivalent to a linear function. As seen in Appendix H, the parametric values in the power-law for each model and dataset, showing γ values larger than 1, even though the fitting algorithm minimizes the same error as regression.

In addition, while 9 points for 3-parameter functions is not a lot, it is also not that little, and, as explained in Appendix H, we further ran bootstrap samples of the points to improve the robustness.

Moreover, given your suggestion, we added simpler function analysis in Appendix K. In this section, we compared the power-law function to simpler two-parameter functions: linear, logarithmic, and exponential functions. We show that while the power-law describes the relationship best, the coupling between behavioral and aggregated internal agreement is somewhat detected in other functions as well. This result indicates the robustness of the relationship across functions, and because the power-law function has the best correlation, it can be a way to describe the relationship.

Could the internal lag simply come from the fact that the two groups have different conversation histories, rather than from a real delay in internal agreement?

Thank you for this comment. We added Appendix O to evaluate this. In this analysis, the reference agreement conversations start at disagreement, similar to the opinion-change test examples. This modification makes the reference group more similar to the opinion-change test group. In this section, we continue to see the lag, further justifying our paper's claim.


We appreciate your comments! We have added the suggested validations in line with your recommendations. We are happy to discuss any further questions.

Dear Reviewer bno6,

We hope that our clarifications and additional results address your concerns. Please let us know if there are any additional questions you may have.

Thank you,

Authors

Thank you for providing clarifications, I am raising my score

Thank you. We’re happy we were able to address your questions.

Reviewer_FGKFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates agreement dynamics in multi-turn conversations between LLM agents from both behavioral and internal representation perspectives. The authors identify an agreement-related direction using a linear probe on hidden activations and observe substantial variability across individual conversations and conversational turns. By aggregating observations across conversations, they show that behavioral agreement rates can be approximated by a first-order Markov model, while aggregated internal agreement can be related to behavioral agreement through an empirical power-law mapping. The paper further reports that internal representations remain distinguishable from reference agreement states after behavioral opinion changes, suggesting a lag between behavioral and internal agreement. These findings are evaluated across three open-weight models and five datasets, with additional robustness and control analyses.

Critical strengths

1. An interesting perspective on agreement dynamics across different levels of analysis.

The paper examines agreement at both the individual-conversation and population levels. The observation that highly variable individual internal trajectories can exhibit more regular patterns after aggregation offers an interesting perspective on multi-agent LLM interactions. This distinction is central to the paper and may provide a useful perspective for studying other collective behaviors in LLM systems.

2. Connecting behavioral agreement with internal representations.

The paper goes beyond analyzing agreement solely through model outputs by extracting agreement-related directions from hidden activations and examining their relationship with behavioral dynamics. The experiments across multiple models and datasets, together with the random-label control and the analysis of post-opinion-change representations, provide useful empirical evidence for this connection.

Critical weaknesses

1. The aggregate agreement curves could be driven by a subset of conversations rather than reflecting a broadly shared trend.

I accept the paper's framing that regularities emerge at the aggregate level. However, Section 3 shows that cross-conversation variance is large, which raises a composition concern: the smooth exponential rise in aggregated agreement (Sections 4 and 5) is equally consistent with most conversations gradually converging, or with a mixture where some conversations converge quickly and others never do. These two scenarios imply very different underlying dynamics but produce the same pooled curve, and the pooled MLE transition estimates in Eq. (4) cannot distinguish them. Could the authors show that the trend is broad-based, e.g., median trajectories alongside means, quantile bands for z̄(t), and the distribution of per-conversation time-to-agreement or transition statistics? This would substantially strengthen the paper's central emergence claim.

2. The power-law mapping in Section 5.1 is not justified against simpler alternatives.

The functional form appears to be chosen by inspection, and no baseline comparison is provided. Since both p(t) and z̄(t) trend monotonically with turn number, some monotone relationship would appear under almost any smooth mapping, and three parameters over roughly eight turns leave considerable freedom. The random-label control validates the agreement direction but not the functional form. The low normalized R-squared values in Table 2 are attributed to flat trajectories, but this arguably means the coupling claim is vacuous in exactly those cells. I would ask the authors to compare the power law against linear and other simple two-parameter mappings and show it fits meaningfully better, especially given that the internal-to-behavioral coupling is a central claim of the paper.

Questions for authors

Q1. To what extent do the aggregate-level agreement regularities reflect the temporal evolution of typical individual conversations?

Could the authors compare ensemble-mean and median trajectories, report quantile bands for internal agreement, and examine the distribution of conversation-level time-averaged agreement rates? It would also be useful to quantify the fraction of individual conversations that exhibit trends consistent with the aggregate trajectory. These analyses would help clarify whether the observed regularities are broadly representative of individual conversational dynamics or primarily emerge through aggregation.

Q2. To what extent does the population-level Markov approximation characterize individual conversational dynamics?

Could the authors examine whether transition probabilities vary across conversational turns and whether substantial heterogeneity exists in transition behavior across conversations? It would also be informative to compare conditional transition probabilities given one versus two preceding states and evaluate whether the fitted Markov model reproduces trajectory-level statistics, such as agreement persistence and run-length distributions, rather than only the marginal agreement probability.

Q3. How robust and specific is the proposed power-law mapping?

Could the authors compare the power-law mapping with linear and alternative nonlinear functions using held-out data? In addition, can a mapping fitted under one experimental condition generalize to another, or does each model–dataset combination require a separately fitted mapping? Such comparisons would help establish whether the observed power-law relationship reflects a robust regularity or a condition-specific empirical fit.

Q4. Could the reported internal agreement lag partly reflect differences in conversational history?

In Section 5.2, the reference agreement representations are drawn from conversations in which the models agree from the beginning, whereas the post-opinion-change representations follow a history of disagreement. Could the authors clarify whether the observed distributional differences reflect delayed internal agreement or persistent encoding of different conversational histories? A history-matched control would help distinguish between these explanations.

2 comments

We thank the reviewer for recognising our novel perspective on understanding dynamics in LLM conversations, which can apply to other collective behaviors. We view this as our main contribution.

Below, we address your main concerns.


The aggregate agreement curves could be driven by a subset of conversations rather than reflecting a broadly shared trend. To what extent do the aggregate-level agreement regularities reflect the temporal evolution of typical individual conversations?

We thank the reviewer for this comment. We added Appendix M to show that internal agreement trajectories mostly form a single cluster that increases in agreement. We additionally show the overall conversation trajectory via an evaluation using median trajectories alongside means; quantile bands for z̄(t) indicate that the overall trend of the internal agreement mostly increases with the increase in turn number.

The power-law mapping in Section 5.1 is not justified against simpler alternatives.

We thank the reviewer for this comment. The power-law experiment aims not only to detect correlation but also to identify a specific function that captures it. Thus, given that correlation, regardless of form, can already indicate a possible relationship between the Markov and internal agreement, we added the Spearman correlation results at the start of Section 5.1. The results show a high correlation between the internal agreement and the Markov behavioral agreement (mean Spearman correlation across datasets = 0.85/0.95/0.93 for Ministral/Qwen/Gemma; p<0.05 in 14 of 15 model--dataset pairs). We also calculated Pearson correlation, which showed high correlation but lower than Spearman (mean Pearson correlation across datasets = 0.78/0.82/0.91 for Ministral/Qwen/Gemma; p<0.05 in 14 of 15 model--dataset pairs). Because Pearson is lower than Spearman, this suggests some non-linearity in the correlation. Note that in the power-law function, γ=1 is equivalent to a linear function, as seen in Appendix H, the parametric values in the power-law for each model and dataset, showing γ values larger than 1, even though the fitting algorithm minimizes the same error as regression.

In addition, while 9 points for 3 parametric functions is not a lot, it is also not that little, and, as explained in Appendix H, we further ran bootstrap samples of the points to improve the robustness.

Moreover, given your suggestion, we added simpler function analysis in Appendix K. In this section, we compared the power-law function to simpler two-parameter functions: linear, logarithmic, and exponential functions. We show that while the power-law describes the relationship best, the coupling between behavioral and aggregated internal agreement is somewhat detected in other functions as well. This result indicates the robustness of the relationship across functions, and because the power-law function has the best correlation, it can be a way to describe the relationship.

To what extent does the population-level Markov approximation characterize individual conversational dynamics?

In Section 3, we detected the high variance in individual conversations' internal agreement; therefore, our work transitioned to evaluating agreement via an aggregated view. In Section 4, we show that aggregated behavioral agreement can be approximated by a first-order Markov model.

To evaluate one versus two preceding history states, in this section, we further show that a second-order Markov model shows a small improvement to the approximation, indicating that a first-order Markov model is a good approximation of the behavioral agreement.

To evaluate whether the transition probabilities vary across turns, we added Appendix G. There, instead of using one transition matrix for all the conversation turns, we use a specific one-per-turn transition matrix, showing consistent results with the main results in the paper.

How robust and specific is the proposed power-law mapping?

Thank you for this comment. We added an evaluation in Appendix L of the generalization of the power-law mapping between models and datasets, showing some generalization in the same model across datasets.

Could the reported internal agreement lag partly reflect differences in conversational history?

Thank you for this comment. We added Appendix O to evaluate this. In this analysis, the reference agreement conversations start at disagreement, similar to the opinion-change test examples. This modification makes the reference group more similar to the opinion-change test group. In this section, we continue to see the lag, further justifying our paper's claim.


Thank you for your thoughtful and positive review! We added additional experiments in line with your recommendations. We are happy to address any additional questions you may have.

Dear Reviewer FGKF,

We hope that our clarifications and additional results address your concerns. Please let us know if there are any additional questions you may have.

Thank you,

Authors

Reviewer_8XRSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies agreement in multi-turn conversations between two instances of the same language model, initialized with opposing personas and asked to discuss binary subjective choices. Across three open-weight model families and five datasets, it trains a linear probe for agreement and finds substantial variability in its scores within and across individual conversations. It then shows that population-level behavioral agreement can be approximated by a two-state first-order Markov model, and fits a power-law mapping from this model’s predicted agreement probability to the population-average activation projection. Finally, it reports that representations following an opinion change gradually approach those of conversations that agreed from the outset, interpreting this as internal agreement lagging behind behavioral agreement.

Critical strengths
  1. The paper asks a relevant question about the relationship between conversational behavior and internal representations.
  2. The experimental setting provides meaningful breadth while remaining tractable.
Critical weaknesses
  1. The activation projection is not independently validated as a measure of agreement: The paper introduces an “internal agreement” direction, but I could not find held-out agreement-classification accuracy, balanced accuracy, or AUROC for that probe. The AUROC analysis in Sec 5.2 instead compares two distributions of projection scores; it does not establish predictive performance against agreement labels.
  2. Agreement-curve prediction does not establish the Markov property: A first-order Markov claim concerns P(st+1∣st,st−1,…)=P(st+1∣st),P(s_{t+1}\mid s_t,s_{t-1},\ldots)=P(s_{t+1}\mid s_t), and the fixed transition matrix additionally assumes time homogeneity. Figure 3 instead evaluates how well the model reproduces the marginal mean E[st]E[s_t] at each turn. History-dependent processes, heterogeneous conversation-specific transition rates, and time-varying processes can produce similar mean curves.

The comparison with a second-order model is helpful, but comparing their aggregate MAEs does not determine whether earlier history improves conditional next-state prediction. Likewise, the statement in Appendix L that agreement depends mostly on the most recent turn despite full-history context is not established by these experiments.

Questions for authors

See above.

6 comments

We thank the reviewer for acknowledging the relevance and timeliness of our research, and the breadth of our experimental settings.


The activation projection is not independently validated as a measure of agreement

Thank you for this comment. We added an analysis of the internal agreement classifier AUROC and a visualization of the separation of agree and disagree labels in Appendix E. We show that the direction separates agreement from disagreement well above chance for all models: the AUROC is 0.74 for Ministral, 0.84 for Qwen, and 0.99 for Gemma. This analysis indicates the usability of our agreement direction.


Agreement-curve prediction does not establish the Markov property

We would like to clarify that our analysis shows that aggregated behavioral agreement is approximated by the Markov model. To make it clearer, we have changed the following lines: Specifically, we changed this line in the intro (line 52): “From a behavioral perspective, we model aggregated convergence towards agreement in model outputs as a first-order Markov process” And in the limitations section: “... a first-order Markov model approximates aggregated behavioral agreement well”.

In addition, to validate the fixed Markov transition matrix, we added Appendix G that evaluates whether using an individual transition matrix per turn improves the predicted aggregated agreement. This setup shows only mild overall improvement over one transition matrix for all turns. This indicates that using the one transition matrix is a good approximation of evaluating the aggregated agreement trends.


We appreciate your comments. We hope our clarifications and additional results fully answer your questions. We are happy to discuss further questions.

Dear Reviewer 8XRS,

We hope that our clarifications and additional results address your concerns. Please let us know if there are any additional questions you may have.

Thank you,

Authors

Thank you for adding held-out agreement-label AUROCs. This addresses my original request for probe validation. Several central interpretations rely on stronger Markovian claims that the experiments do not establish. In particular, aggregate mean prediction does not imply that individual agreement depends mostly on the most recent turn, and a power-law mapping from p(t)p(t) to zˉ(t)\bar{z}(t) does not establish Markovian dynamics in internal representations.

Therefore, I am not convinced that the claims of this paper is fully solid, and I lean toward remaining my score.

Comments on the paper

1 comment

We thank the reviewers for their comments and for suggesting additional experiments. We added additional experiments that further show the robustness of our hypothesis and results:

Appendix E (Reviewers 8XRS, bno6) - An Internal agreement classifier shows high classification accuracy. This indicates that while individual conversations show high variance trends, the agreement direction still detects internal agreement.

Appendix G (Reviewers 8XRS, FGKF) - Using a specific Markov transition matrix for each turn instead of one matrix for all turns results in similar trends.

Appendix K (Reviewers FGKF, bno6) - We compare the coupling between behavioral and internal agreement using simpler functions than power-law. We show that while the power-law describes the relationship best, the coupling is somewhat detected in other functions as well.

Appendix L (Reviewer FGKF) - Evaluates the generalization of the power-law mapping between models and datasets, suggesting some generalization in the same model between datasets.

Appendix M (Reviewer FGKF) - Shows that internal agreement trajectories mostly form a single cluster that increases in agreement.

Appendix O (Reviewers FGKF, bno6)- An ablation analysis where the reference agreement conversations start with disagreement conversations similar to the test-opinion change examples.

We also added Spearman correlation statistics to Section 5.1, showing high correlation between the Markov model and aggregated internal agreement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.