acceptodds
Under review as a conference paper at ICLR 2027

Who Is the Influencer? Consensus Dynamics in Multi-Agent Systems

Abstract

Multi-agent LLM systems are increasingly used in practice, but their behavior is complex and hard to understand. We design a simple task to ask which model has the most influence on the final consensus when several models are mixed in one system. Experimentally, across seven model families, we find that the opinions of the other models are absorbed into the opinion of the stronger model, the one with more parameters. This happens even though the task requires no pretraining knowledge and has no correct answer. To understand this phenomenon, we interpret the agents as weighted particles moving on a simplex. Larger models carry a larger weight and therefore occupy a deeper well in the energy landscape. A large weight means strong in-context learning: a model with a large weight amplifies the majority of its context. Larger models are generally better at in-context learning, and this is what makes the larger model the influencer. We hope that this work helps to interpret multi-agent systems, whose behavior is becoming too complex for humans to understand.

Primary area:
interpretability and explainable AI
Keywords:
multi-agent systems, large language models, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KbezOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates influence and consensus formation in heterogeneous multi-agent LLM systems through a naming game. Experiments across seven model families show that stronger models generally achieve higher consensus win rates. The authors also examine the effects of population size and memory length. To explain these findings, they introduce a model-specific parameter called assertion strength and extend an existing consensus-energy framework using replicator dynamics. The framework is evaluated by fitting assertion strengths and comparing predicted dynamics with observed trajectories in three open-weight model families.

Critical strengths
  1. The naming game with meaningless labels allows the authors to study influence among heterogeneous LLMs without interference from answer correctness.

  2. Experiments across seven model families show that stronger models generally have higher consensus win rates, with additional experiments examining the effects of population size and memory length.

Critical weaknesses
  1. The paper mainly extends the naming-game framework of Tanaka (2026) to heterogeneous LLMs. Given prior findings on LLM conformity and asymmetric influence, the observed results provide limited additional scientific insights.

  2. The connection between the agent-level behavior and the proposed macroscopic dynamics relies on heuristic assumptions. Moreover, Equation (4) can yield invalid probabilities for the reported parameters, and the framework is evaluated using the same experimental runs from which its parameters are fitted. These issues weaken the proposed mechanistic explanation.

Questions for authors
  1. Given prior work on LLM conformity and consensus formation, particularly Tanaka (2026), what new, non-trivial insights does this work provide beyond extending the existing naming-game framework to heterogeneous LLMs and observing higher consensus win rates for stronger models?

  2. How are the macroscopic dynamics in Equation (3) justified by the agent-level model in Equation (4)? In addition, how does Equation (4) ensure valid output probabilities for the reported assertion strengths (e.g., m = (0.9, 0.1, 0), w = 3.44)?

  3. Can assertion strengths be estimated independently and used to predict consensus outcomes on held-out experiments? Are these parameters stable and comparable across different model families?

4 comments
AuthorsOpenReview ↗

Author Response (1/3)

We thank the reviewer for the careful reading and precise questions. Please let us know if our responses in the following address your concerns.

We revised the paper based on the reviewers' comments, and the major edit was highlighted with coloring (red). Please also check the updated manuscript.

W2 & Q3

The framework is evaluated using the same experimental runs from which its parameters are fitted.

Can assertion strengths be estimated independently and used to predict consensus outcomes on held-out experiments? Are these parameters stable and comparable across different model families?

Yes. To address your concern, we now estimate the assertion strengths independently of the runs. Each model is queried alone, with the same prompt as in the game, on synthetic memories that occur in no population run, and the same regression as before is fitted to these isolated answers. No population run enters w, so every run is held out.

With this independent estimate, the results of the submission are reproduced for Gemma 4 and Qwen 2.5. For Gemma 4, the strengths are 2.96 (31B), 2.67 (26B-A4B) and 2.47 (E4B), in the same order as the values fitted on the runs (3.44, 3.19, 2.52). The well of the 31B model is still the deepest, the flow agrees with the measured motion at r = 0.67 (0.65 before), and the energy falls in 86% of the rounds (85% before). The order of w matches the order of win rates in every pair of classes in Gemma 4, Qwen 2.5 and Gemini.

As for stability, the strengths measured alone have the same order as those fitted on the runs for Gemma 4 and Qwen 2.5 and differ from them by at most 0.53 (table in Appendix H), and every pair of their models keeps its order in at least 99% of bootstrap resamples. For Gemini, the two estimates differ: only the one measured alone follows the ladder. Additionally, when the memory states of the runs are replayed to each model alone, the model reproduces what it said in the run: the probability of the majority label agrees within 0.03 for the six open-weight models. As for comparability, all models are measured on the same kind of memories with the same regression, so the strengths share one scale; they range from 1.18 to 3.55 across three families.

Therefore, we believe your concern is resolved: the framework is no longer evaluated on the runs from which its parameters are fitted. In the revised paper, we rewrote Section 4.3 (pages 7–9) around the independently estimated w, redrew Figure 9 and Figures 11–14 (Appendix I, pages 23–25) with it, and described the estimation in Appendix H (pages 22–23). Thank you for this question, which made the paper stronger.

W2 & Q2 (valid probabilities)

Equation (4) can yield invalid probabilities for the reported parameters.

How does Equation (4) ensure valid output probabilities for the reported assertion strengths (e.g., m = (0.9, 0.1, 0), w = 3.44)?

Thank you for pointing this out. You are right: Equation 4 does not ensure this. For your example it gives x = (1.148, −0.148, 0), and it is a valid distribution on the whole simplex only for w ≤ 1.

To address this, we added the rule x_k ∝ m_k exp(w m_k) (Equation 5), which is a valid distribution for every w, and we state on which states Equation 4 is invalid. Equation 4 is the first-order expansion of Equation 5 in w, and both have the same continuous-time limit (Equation 3), so the energy and the flow of Section 4 are unchanged. In the revised paper, the least-squares regression on this first-order term is kept as the estimator of w. The quantities tested in Section 4 (which well is the deepest, the direction of the flow, and whether the energy falls) do not depend on the overall scale of w. When Equation 5 is fitted directly to the utterances of the runs, the order of the strengths within Gemma 4 and within Qwen 2.5 is the same.

In the revised paper, we added the corrected rule as Equation 5 in Appendix H (page 23), together with the fraction of visited states on which Equation 4 is invalid.

AuthorsOpenReview ↗

Author Response (2/3)

W2 & Q2 (from Equation 4 to Equation 3)

The connection between the agent-level behavior and the proposed macroscopic dynamics relies on heuristic assumptions.

How are the macroscopic dynamics in Equation (3) justified by the agent-level model in Equation (4)?

Equation 3 is obtained from Equation 4 under two closure approximations, which are stated in Appendix G: the agents share the population's composition, and the strength acting on a label is that of the class that brought it. We agree that this step is heuristic. For this reason, the paper treats Equation 2 as a posited energy and tests it directly, in the spirit of replicator dynamics [1].

We would like to emphasize that this test is now out of sample. With strengths measured outside the population, the flow of Equation 3 agrees with the measured motion in three families (r = 0.61–0.67), and the energy falls in 84–88% of the rounds. Additionally, the agent-level prediction in Appendix K uses neither Equation 3 nor the closure, and names the large model as the most likely winner in all five families.

To make this clearer, we revised Section 4.3 (pages 7–9) and the paragraph "Measurement" in Appendix J (page 26), and added Appendix K (pages 27–29).

[1] Hofbauer & Sigmund, Evolutionary Games and Population Dynamics. Cambridge University Press, 1998.

AuthorsOpenReview ↗

Author Response (3/3)

W1 & Q1

The paper mainly extends the naming-game framework of Tanaka (2026) to heterogeneous LLMs. Given prior findings on LLM conformity and asymmetric influence, the observed results provide limited additional scientific insights.

We would like to clarify the contribution of this paper relative to the prior work from three perspectives.

  1. Who wins is a measurable property of single models.

Tanaka [1] studies homogeneous populations, in which the winning convention is a lottery. We show that, when models are mixed, the outcome is no longer a lottery (the large model wins most often in every family; Figure 3), and that, in the three families we analysed, the order of win rates follows the order of a strength that is measured on each model alone, without using any population run.

  1. Outcome of a population, not susceptibility of one model.

Conformity studies measure how much one model yields to a majority [2, 3]. They do not say which convention a mixed population adopts. In LLM naming games, a committed minority of scripted agents can overturn a convention [4]; our agents are unscripted and differ only by model.

  1. A mechanism tested by intervention.

We tested whether in-context learning (ICL) ability causes the influence, by an intervention inside one model. In the Qwen 2.5 population, we modified only the large model (32B). First, we computed an induction score for each of its attention heads [5]: on a sequence of random tokens repeated twice, the attention from a token in the second copy to the token that followed its first occurrence. We then zeroed the output of the 128 heads with the highest score. We compared this with two controls. In the first, the intact model is run at the temperature that gives it the same output entropy as the ablated model, so that both answer equally sharply; it wins 29 of 36 trials. In the second, 128 other heads are zeroed, with the same number per layer; it wins 32 of 36 trials. With every output constrained to be a valid label, the model without its induction heads wins 11 of 36 trials (p = 2e-5 against the first control). Without this constraint it wins 6 of 36 trials and produces many invalid outputs. Therefore, the influence of this model is removed by reducing its ICL ability, and not by lowering its sharpness or by removing the same number of other heads. The ablation covers one model of one family. We searched for prior work that links ICL ability to influence in a population and found no experimental test of it. The closest work we found is a position paper that proposes to study multi-agent LLM systems with mechanistic interpretability, and gives attention heads that copy tokens between agents as a hypothetical example, without experiments [6]. Debate studies with a correct answer report that accuracy can fall even when stronger models outnumber weaker ones, because models give up correct answers in response to their peers [7]; they describe the behaviour and do not identify a mechanism.

To make the contribution clearer, we revised the abstract (page 1), the introduction and contributions (pages 2–3), Section 4.3 (pages 8–9) and the conclusion (page 10), and added [6] and [7] to the related work in Section 5 (page 10).

[1] Tanaka, When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs. 2026. https://arxiv.org/abs/2603.24676

[2] Weng et al., Do as We Do, Not as You Think: the Conformity of Large Language Models. ICLR 2025.

[3] Zhu et al., Conformity in Large Language Models. ACL 2025.

[4] Flint Ashery et al., Emergent Social Conventions and Collective Bias in LLM Populations. Science Advances, 2025.

[5] Olsson et al., In-context Learning and Induction Heads. 2022. https://arxiv.org/abs/2209.11895

[6] Lee et al., Towards Ethical Multi-Agent Systems of Large Language Models: A Mechanistic Interpretability Perspective. 2025. https://arxiv.org/abs/2512.04691

[7] Wynn et al., Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate. 2025. https://arxiv.org/abs/2509.05396

Again, we appreciate the detailed comments from the reviewer. Please let us know if our responses address your concerns. If needed, we will be happy to clarify them further.

AuthorsOpenReview ↗

Kind Reminder to Reviewer Kbez

Thank you again for your valuable review and constructive feedback on our paper. As we have not yet received a response, we would like to kindly remind Reviewer Kbez that we have thoroughly addressed your concerns. Specifically, we now estimate the assertion strengths independently of the runs, so that the framework is tested out of sample, and added a single-agent rule that always gives valid probabilities (Equation 5). We have also clarified our contribution relative to Tanaka (2026) and conducted an additional intervention experiment involving induction-head ablation. These updates have been incorporated into the revised paper, which we believe has been significantly strengthened as a result. We would greatly appreciate it if you could review these updates and kindly reconsider your score in light of our revisions to address your comments.

Reviewer_oUKMOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks which model wins when different LLMs must agree as a group. The authors use a simple naming game: agents repeatedly pass meaningless five-letter labels to each other until everyone uses the same one. Each group mixes a large, a medium and a small model from the same family (for example Opus, Sonnet and Haiku), and each model starts with its own label. Since the labels mean nothing, there is no right answer, so the game measures influence alone. Across seven model families, the group most often adopts the label of the larger or stronger model. When models of similar size from different families are mixed, no model clearly wins. To explain this, the authors give each model a pull strength, estimated from the runs, that measures how strongly it repeats the majority of what it has heard. Larger models pull harder, and the authors suggest this is because larger models are better at learning from their context.

Critical strengths

The question is simple and useful: when a system mixes cheap and expensive models, whose view wins? The game is well designed to answer it, because meaningless labels remove any effect of what the models already know. The results hold across seven families, including commercial and open models, and the statistics are careful. The authors are also open about where their explanation fails. The paper is clearly written, and the figures make the dynamics easy to follow.

Critical weaknesses

The finding is clear, but the explanation is not convincing. The authors measure each model's pull strength from the same games they then use to explain, so the match is built in rather than tested (they admit this themselves). The explanation also covers only the three open-model families. The claim that bigger models win because they learn better from their prompt is never tested. Simpler reasons could give the same result: bigger models may just answer less randomly, or smaller models may give more broken answers, which the game counts as silence. The paper also treats larger and better as the same thing. Its one test that separates them suggests benchmark quality does not matter much: Gemma 31B is much better than Qwen 2.5 32B on benchmarks but barely wins more often. Finally, the game is very artificial (single made-up words, equal groups, no right answer), so it is unclear what it tells us about real multi-agent systems.

Questions for authors

If a small model is made to answer less randomly (e.g., with a lower temperature), does it gain influence? This would separate "less random" from "better at learning from context". Can the pull strength be measured from models queried alone, as in Figure 10, and then used to predict the winner of new runs?

4 comments
AuthorsOpenReview ↗

Author Response (1/3)

We thank the reviewer for the constructive feedback. Please let us know if our responses in the following address your concerns.

We revised the paper based on the reviewers' comments, and the major edit was highlighted with coloring (red). Please also check the updated manuscript.

W1 & Q2

The authors measure each model's pull strength from the same games they then use to explain, so the match is built in rather than tested.

Can the pull strength be measured from models queried alone, as in Figure 10, and then used to predict the winner of new runs?

Yes. To address your concern, we now measure the pull strength from models queried alone, as you suggested, and we no longer fit it on the games. Each model is queried with the same prompt as in the game, on synthetic memories that occur in no population run, and the same regression as before is fitted to these isolated answers. No population run enters the strength.

With this isolated strength, the results of the submission are reproduced for Gemma 4 and Qwen 2.5. For Gemma 4, the strengths are 2.96 (31B), 2.67 (26B-A4B) and 2.47 (E4B), in the same order as the values fitted on the games (3.44, 3.19, 2.52). The well of the 31B model is still the deepest, the flow agrees with the measured motion at r = 0.67 (0.65 before), and the energy falls in 86% of the rounds (85% before).

As for the winner of runs that were not used, the order of the isolated strengths matches the order of win rates in every pair of classes in Gemma 4, Qwen 2.5 and Gemini. Additionally, simulating the game agent by agent from the isolated answers names the large model as the most likely winner in all five families we tested.

Therefore, we believe your concern is resolved: the match is no longer built in. In the revised paper, we rewrote Section 4.3 (pages 7–9) around the isolated strength, redrew Figure 9 and Figures 11–14 (Appendix I, pages 23–25) with it, and added the measurement to Appendix H (pages 22–23) and the prediction to Appendix K (Table 4, page 27). Thank you for this suggestion, which made the paper stronger.

W2

The explanation also covers only the three open-model families.

To address this concern, we applied the same isolated measurement to an API family, Gemini, which is possible now that the strength does not have to be fitted on the games. For Gemini, the order of the strengths matches the order of win rates in all three pairs of classes, the flow agrees with the measured motion at r = 0.63, and the energy falls in 84% of the rounds. Additionally, the agent-level simulation in Appendix K gives large > medium > small for GPT-5, Claude 5 and Gemini.

Therefore, the explanation is no longer restricted to the open-model families. In the revised paper, we added the Gemini results to Appendix I (Figures 11–14, pages 23–25) and the agent-level prediction to Appendix K (Table 4, page 27). Thank you for raising this point.

AuthorsOpenReview ↗

Author Response (2/3)

W3 & Q1

The claim that bigger models win because they learn better from their prompt is never tested. Simpler reasons could give the same result: bigger models may just answer less randomly.

If a small model is made to answer less randomly (e.g., with a lower temperature), does it gain influence?

To address your concern, we conducted the following two experiments.

  1. A small model that answers less randomly (your suggestion):

We ran only the small model at temperature 0.3, and predicted the outcome before running it. The prediction was made as follows. For the open-weight models, the isolated measurement gives the log-probability of each label for each memory. From these log-probabilities, the answer distribution of the small model at temperature 0.3 can be computed directly, without any new run: each probability is raised to the power 1/0.3 and renormalized. We then simulated the game agent by agent, with the small model answering from this sharpened distribution and the large and medium models answering from their unchanged distributions. The predicted win rates were written to a file and fixed before the population was run.

Small model at T = 0.3predictedobserved small-class wins
Gemma 4 E4Bgain (.31 → .42)8 → 22 of 108
Qwen 2.5 7Blittle change (.01 → .07)10 → 10 of 108

Answering less randomly raises influence in Gemma 4, as predicted (a gain of .11 predicted, .13 observed), although the level of small-class wins is overpredicted both before and after the intervention. In Qwen 2.5 little change was predicted, and the small model's wins do not change (10 before and after). So answering less randomly is not sufficient by itself.

  1. A large model whose in-context learning (ICL) ability is removed, at fixed randomness:

We removed the 128 attention heads with the highest induction score [1] from Qwen 2.5 32B. With every output constrained to be a valid label, it wins 11/36. The intact model, run with the same constraint at the temperature that gives it the same output entropy, wins 29/36 (p = 2e-5), and removing 128 random heads leaves 32/36. Without the constraint, the ablated model wins 6/36 and produces many invalid outputs.

Therefore, we believe your concern is resolved: "answers less randomly" and "learns from its context" are now separated for this model, and its ICL ability is necessary for its influence. The ablation covers one model of one family. In the revised paper, we added the first experiment to Appendix K (Table 6, page 29) and the second to Section 4.3 (pages 8–9) and Appendix L (Table 7, page 29). Thank you for proposing this experiment.

[1] Olsson et al., In-context Learning and Induction Heads. 2022. https://arxiv.org/abs/2209.11895

W4

Smaller models may give more broken answers, which the game counts as silence.

To address this concern, we counted the invalid answers of every model in the main runs. The rate is zero for every small model, and for every medium model except Claude Sonnet 5 (3 of 41,233 probes). The only model with a non-negligible rate is Claude Opus 5, the large model of its family (3.9% of probes), and it still wins most often. Additionally, in the ablation above we forced every output to be a valid label, and the effect remained (11/36 against 29/36, p = 2e-5).

Therefore, broken answers do not explain why the smaller models lose. In the revised paper, we added these rates to Appendix B (page 19) and a pointer to them in Section 3.1 (page 5). Thank you for pointing out this alternative.

W5

The paper also treats larger and better as the same thing ... Gemma 31B is much better than Qwen 2.5 32B on benchmarks but barely wins more often.

We agree with your reading of that test, and we would like to clarify that it is consistent with our account. What sets the influence of a model is how it responds to its memory, which we now measure outside the population. Benchmark quality does not enter. Gemma 4 31B and Qwen 2.5 32B differ widely on MMLU-Pro (85.2 against 69.0), but their measured strengths are similar (2.96 and 3.10), and they win about equally often when they are mixed (Fig. 6).

To make this clearer, we added the paragraph "Larger, better, and what predicts the winner" to Appendix M (page 30) and revised the abstract (page 1), which no longer equates the stronger model with the one that has more parameters.

AuthorsOpenReview ↗

Author Response (3/3)

W6

The game is very artificial (single made-up words, equal groups, no right answer).

We would like to emphasize that the setting is minimal by design. Meaningless labels are what allow influence to be measured without correctness, and the naming game is the standard model of convention formation in both human and LLM populations [1, 2, 3]. Within this setting, the main result is aggregated over six label triples. In Gemma 4, the large model wins most often at every population size (N = 3 to 24) and memory length we tested. It also wins most often when the small model is the minority or is run at a lower temperature. When the large model itself is reduced to 4 of 24 agents, it never wins, as the isolated measurements predict: influence depends on the numbers as well as on the model. We do not claim that the same model wins in tasks with a correct answer or on structured networks, where correctness adds a pull of its own [4].

To make this clearer, we added the paragraph "How minimal the game is" to Appendix M (page 30) and the unequal-group results to Appendix K (Table 6, page 29).

[1] Centola et al., Experimental Evidence for Tipping Points in Social Convention. Science, 2018.

[2] Flint Ashery et al., Emergent Social Conventions and Collective Bias in LLM Populations. Science Advances, 2025

[3] Tanaka, When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs. 2026. https://arxiv.org/abs/2603.24676

[4] El et al., Physics of Agents: Statistical Mechanics Predicts Collective Behavior of AI Agents. 2026. https://arxiv.org/abs/2608.16578

Again, we appreciate the detailed comments from the reviewer. Please let us know if our responses address your concerns. If needed, we will be happy to clarify them further.

AuthorsOpenReview ↗

Kind Reminder to Reviewer oUKM

Thank you again for your valuable review and constructive feedback on our paper. As we have not yet received a response, we would like to kindly remind Reviewer oUKM that we have thoroughly addressed your concerns. Specifically, we now measure the pull strength from models queried alone, as you suggested, and conducted additional experiments involving a small model at a lower temperature (predicted before it was run) and an induction-head ablation with an entropy-matched control. We have also reported the rate of invalid answers for every model and clarified the distinction between larger and better models. These updates have been incorporated into the revised paper, which we believe has been significantly strengthened as a result. We would greatly appreciate it if you could review these updates and kindly reconsider your score in light of our revisions to address your comments.

Reviewer_6azyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks the question of which model us best able to steer a swarm of LLM agents when there is a clear capacity mismatch and where there is no objective 'correct' answer to the questions being asked. The authors use a naming game and group different families that has 3 diosticnt model capacities. The paper finds that in tasks where the knowledge of the model plays little to no role, the larger and styronmger model's bias will herd the entire group of agents towards concensus with the largest model. The authors validate this across OpenAI, Claude, Gemini, and open weight model families. Contrastingly 3 similar-sized models from different families tie near chance. The authors conjecture that ww reflects in-context learning ability.

Critical strengths

The authors have done a good job designing the experiment so that correctness and influence are separated, allowing the paper to focus on the bias that exists between the smaller and larger models. The labels carry no inherent semantic meaning, and the prompts provided to the models never ask for agreement. All the studied classes start from symmetry on their own labels. The bias that survives is hence a property of the model itself and not any external signals that it may have received. Therefore in this setting the ordering of large to medium to small sppear in all of the seven familes of models studied.

Critical weaknesses

My main issue is that Section 4 does not actually explian anything. If each model has its own w, then the class with the biger w will win. This can be inferred directly from Eq. 4 without any energy or replicator machinery. And w is fit from the same runs it is then compared against, so the nine of nine ordering in Fig. 14 is not really a test of anything, rather just another diagnositic of the behaviour that we have already observed in the consensus behaviour. The energy itself has a problem the authors flag in App. J. It only picks a winner because each label is tied to the model that introduced it, and the version without that trick cannot tell the wells apart. It also only works for the three open weight families, the API ones cannot be fit at all. What I actually wanted to know is why w is bigger for bigger models (or at least hypotheses towards this idea), and the paper just says in-context learning and moves on. I am not convinced it is anything more than the bigger models sampling with lower entropy. The decoding temperature is not even stated, so it is hard to infer all the needed information from the current submission.

Questions for authors
  1. Can w be measured outside the population, and does it still predict the winner?

  2. Is assertion strength in-context learning or just output sharpness?

5 comments
AuthorsOpenReview ↗

Author Response (1/2)

We thank the reviewer for the careful reading and constructive feedback. Please let us know if our responses in the following address your concerns.

We revised the paper based on the reviewers' comments, and the major edit was highlighted with coloring (red). Please also check the updated manuscript.

W1 & Q1

w is fit from the same runs it is then compared against, so the nine of nine ordering in Fig. 14 is not really a test of anything.

Can w be measured outside the population, and does it still predict the winner?

Yes, w can be measured outside the population, and the results of Section 4 still hold with it. To address your concern, we no longer fit w on the runs. Instead, we query each model alone, with the same prompt as in the game, on synthetic memories that occur in no population run, and fit the same regression as before to these isolated answers. No population run enters w.

With this isolated w, the results of the submission are reproduced for Gemma 4 and Qwen 2.5. For Gemma 4, the strengths are 2.96 (31B), 2.67 (26B-A4B) and 2.47 (E4B), in the same order as the values fitted on the runs (3.44, 3.19, 2.52). The well of the 31B model is still the deepest. The flow agrees with the measured motion at r = 0.67 (0.65 with the fitted w), and the energy falls in 86% of the rounds (85% before). Qwen 2.5 behaves in the same way.

As for the winner, the order of the isolated w matches the order of win rates in every pair of classes in Gemma 4, Qwen 2.5 and Gemini. Additionally, we simulated the game agent by agent from the isolated answers. This predicted, before the run, that a large class reduced to 4 of 24 agents never wins, and it won 0 of 72 trials.

Therefore, we believe your concern is resolved. In the revised paper, we rewrote Section 4.3 (pages 7–9) around the isolated w, redrew Figure 9 and Figures 11–14 (Appendix I, pages 23–25) with it, and described the measurement in Appendix H (pages 22–23). Thank you for this suggestion, which made the paper stronger.

W2

It also only works for the three open weight families, the API ones cannot be fit at all.

To address this concern, we applied the same isolated measurement to an API family, Gemini. Fitted on the runs, w does not follow the Gemini ladder (3.99, 3.95, 4.15). Measured alone, it does (3.55, 3.42, 2.86).

For Gemini, the order of w matches the order of win rates in all three pairs of classes, the flow agrees with the measured motion at r = 0.63, and the energy falls in 84% of the rounds. Additionally, we simulated the game agent by agent from each model's isolated answers for GPT-5, Claude 5 and Gemini. This gives large > medium > small in all three API families, and for GPT-5 it names the most frequent winner in 14 of 16 label assignments. For GPT-5 this prediction uses each model's preference for particular label words. We added this term after a diagnostic on the runs showed that the label assignment matters, and then measured it on each model alone (Appendix K).

Therefore, the account is no longer restricted to the open-weight families. In the revised paper, we added the Gemini results to Appendix I (Figures 11–14, pages 23–25) and the agent-level prediction to Appendix K (Table 4, page 27). Thank you for raising this point.

AuthorsOpenReview ↗

Author Response (2/2)

W3 & Q2

I am not convinced it is anything more than the bigger models sampling with lower entropy.

Is assertion strength in-context learning or just output sharpness?

To address your concern, we conducted an intervention that separates the two inside one model. We prepared the following variants of the large model in the Qwen 2.5 population:

  1. Induction heads removed: the 128 attention heads with the highest induction score [1] are zeroed.
  2. Entropy-matched control: the intact model is run at the temperature that gives it the same output entropy as variant 1, so it is equally "unsharp".
  3. Random heads removed: 128 other heads are zeroed, with the same number per layer.
Large model of the Qwen 2.5 populationlarge-class wins
intact85/108
entropy-matched control29/36
random heads removed32/36
induction heads removed6/36
induction heads removed, every output forced to be a valid label11/36

The cleanest comparison is the one in which every output is a valid label in both arms: the model without its induction heads wins 11/36, against 29/36 for the entropy-matched control (p = 2e-5). Without this constraint the ablated model wins 6/36 (p = 4e-8), but it then also produces many invalid outputs. Lowering the sharpness of the intact model to the same entropy does not remove its advantage. Therefore, output sharpness alone does not explain the influence of this model, and its in-context learning (ICL) ability is necessary for it.

We also note the limitations. The ablation covers one model of one family. Across the 18 models, the fitted strength is higher when the output entropy is lower (Spearman ρ = −0.80), and its partial correlation with behavioural in-context scores is not significant, which is why we needed an intervention and not a correlation.

In the revised paper, we added this experiment to Section 4.3 (pages 8–9) and Appendix L (Table 7, page 29). Thank you for this question. The in-context account is now tested for this model, and no longer only suggested.

[1] Olsson et al., In-context Learning and Induction Heads. 2022. https://arxiv.org/abs/2209.11895

W4

Section 4 does not actually explain anything. If each model has its own w, then the class with the bigger w will win. This can be inferred directly from Eq. 4 without any energy or replicator machinery.

We would like to clarify that this does not follow from Equation 4 alone. Equation 4 does not depend on the label. An agent with a large w amplifies whichever label is the majority of its own memory, and once the memories mix, this includes the labels of the other classes. This is the conformist transmission of Boyd & Richerson [1], which favours whatever is locally common. Which label finally wins is therefore decided by a race between mixing and amplification from the initial state.

The energy and the flow describe this race: where the population ends and in which direction it moves. With w measured outside the population, the flow agrees with the measured motion in three families (r = 0.61–0.67), and the energy falls in 84–88% of the rounds.

To make this point clearer, we added it to Section 4.2 (page 7).

[1] Boyd & Richerson, Culture and the Evolutionary Process. University of Chicago Press, 1985.

W5

The energy ... only picks a winner because each label is tied to the model that introduced it, and the version without that trick cannot tell the wells apart.

We agree with this limitation of the energy. The agent-level prediction in Appendix K does not use this tie: each agent is distinguished only by its model and its initial memory, and the prediction names the large model as the most likely winner in all five families.

To make this clearer, we revised Appendix J (page 26) and added Appendix K (pages 27–29).

W6

The decoding temperature is not even stated.

Thank you for pointing this out. All models are sampled at temperature 1.0, except the GPT-5 family, which does not accept this parameter and runs at the provider's default. We also use the lowest reasoning effort, at most 256 output tokens, and up to 8 retries.

In the revised paper, we explicitly stated these settings in Section 2.1 (page 3) and Appendix A (page 18).

Again, we appreciate the detailed comments from the reviewer. Please let us know if our responses address your concerns. If needed, we will be happy to clarify them further.

AuthorsOpenReview ↗

Kind Reminder to Reviewer 6azy

Thank you again for your valuable review and constructive feedback on our paper. As we have not yet received a response, we would like to kindly remind Reviewer 6azy that we have thoroughly addressed your concerns. Specifically, we now measure the assertion strength w outside the population and show that it still predicts the winner, extended the energy analysis to an API family (Gemini), and conducted an induction-head ablation with an entropy-matched control to separate in-context learning from output sharpness. We have also stated the decoding temperature and settings. These updates have been incorporated into the revised paper, which we believe has been significantly strengthened as a result. We would greatly appreciate it if you could review these updates and kindly reconsider your score in light of our revisions to address your comments.

Comments on the paper

1 comment
Area_Chair_zySnOpenReview ↗

Respond to rebuttal

Dear reviewers,

as the discussion phase with the authors is coming to an end in a few hours, please answer to the rebuttal and summarize the reasons for your score decision.

Thanks a lot, the AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.