acceptodds
Under review as a conference paper at ICLR 2027

Strategic Diversity Does Not Scale: The Bounded Geometry of LLM Collective Behavior

Abstract

Multi-model AI systems are often motivated by an expectation that independently developed language models provide complementary judgments. We examine this expectation using 50 LLMs from 26 model families evaluated on 20 strategic-game conditions spanning signaling, coordination, Bayesian reasoning, and deception. We represent model responses using text embeddings and study the geometry of both individual responses and model-level behavioral signatures. As the evaluated ecosystem grows, we observe that dimensionality of the pooled response space increases with sharply diminishing marginal returns. At 50 models, principal component analysis (PCA) over model-level signatures requires 16 components to explain 90% of between-model variance. Model signatures also exhibit high pairwise similarity and substantial cross-family geometric alignment. These patterns persist in action-only representations, alternative dimensionality estimators, and an open-ended generation dataset. Together, our results indicate that nominal growth in models and providers produces less behavioral expansion than model count alone would suggest. We further investigate strategic diversity when models are aggregated following three patterns: majority vote, multi-agent discussion, cross provider composition. Majority voting reinforces model's dominant responses. Multi-agent discussion among diverse models produces no clear gain in strategic score, while critique from another provider often changes the decision but does not reliably improve it. Effective collective systems thus require measured coverage of useful behaviors, not simply more models, providers, or calls.

Primary area:
foundation or frontier models, including LLMs
Keywords:
strategic games, large language models, geometry, multi-model system

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wsTxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether adding more LLMs increases behavioral diversity in strategic reasoning. The authors evaluate 50 LLMs (26 model families) across 20 strategic games (signaling, coordination, Bayesian reasoning, and deception). They use text embeddings, Principal Component Analysis (PCA), and bottleneck autoencoders to measure behavioral similarity and dimensionality. The results show diminishing returns as more models are added, with 16 principal components explaining 90% of between-model variance and autoencoder reconstruction plateauing at 18-24 dimensions. The authors also evaluate three aggregation methods (majority voting, multi-agent discussion, and cross-provider critique), finding that majority voting improves average strategic scores (0.492 to 0.600), while the other two do not consistently improve performance.

Critical strengths
  1. The paper provides a large-scale empirical analysis across 50 LLMs (26 model families) and 20 strategic-game conditions. The distinction between pooled response variation and between-model variation is useful, with PCA requiring 137 dimensions to explain 90% of pooled response variance but only 16 dimensions for between-model variance. The diminishing growth in dimensionality (76.2 at 5 models to 136.2 at 50 models) provides evidence that adding models contributes progressively less to the measured behavioral space. I find this useful for understanding diversity in multi-model systems.

  2. Majority voting increases the average strategic score from 0.492 to 0.600, while cross-provider critique changes decisions in 40.2% of cases but improves scores in only 4.6%. I find this distinction between changing decisions and improving decision quality particularly interesting. The authors also conduct robustness checks using alternative embeddings, action-only representations, autoencoder architectures, and open-ended generation, which provide additional support for the observed geometric patterns.

Critical weaknesses
  1. I find the motivation and significance somewhat limited given existing work on LLM behavioral homogeneity, correlated errors, and strategic reasoning [1, 2, 3]. The paper introduces a distinct geometric analysis using PCA and autoencoders to study how behavioral dimensionality changes as more models are added. However, it remains unclear what additional insight these findings provide beyond the already established observation that LLMs exhibit similar behaviors and correlated errors. I think the authors should demonstrate whether selecting geometrically diverse models improves collective strategic performance compared with selecting similar or randomly chosen models, which would establish the practical value of the proposed analysis.

  2. The authors show that 16 principal components explain 90% of between-model variance, but this does not necessarily imply that models make similar strategic decisions or share the same errors. For example, two models may produce similar reasoning but choose different actions, or provide different explanations while making the same incorrect decision. While the action-only analysis supports the geometric findings, I think a direct analysis of the relationship between geometric distance, action disagreement, and error correlation is needed to establish whether the observed similarity reflects actual strategic behavior.

  3. The experiments use a fixed generation temperature (0.7) and a standardized zero-shot prompt, without examining how different temperatures or prompting methods affect the measured behavioral diversity.

  4. Prior work such as Social Agents [4] shows that demographic and psychographic persona conditioning can produce diverse behavioral predictions and outperform repeated calls to the same model without personas. This paper only examines diversity across different LLMs and does not investigate whether assigning distinct personas to the same model can produce additional strategic diversity. I think comparing different persona-conditioned agents, repeated calls without personas, and combinations of models from different providers under the same games and computational budget would be useful. If persona conditioning produces greater diversity or better strategic decisions, it would suggest that strategic diversity depends on how agents are constructed rather than simply how many models are combined, limiting the generality of the paper's conclusion.

[1] Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond). NeurIPS 2025. [2] Correlated Errors in Large Language Models. ICML 2025. [3] Playing Repeated Games with Large Language Models. Nature Human Behaviour, 2025. [4] Social Agents: Collective Intelligence Improves LLM Predictions. ICLR 2026.

Questions for authors
  1. Could the authors clarify what additional insights the geometric dimensionality analysis provides beyond existing studies of LLM behavioral homogeneity and correlated errors [1, 2, 3]?

  2. Have the authors examined whether geometric distance between models is related to action disagreement or error correlation?

  3. Were different generation temperatures or prompt formulations evaluated? If so, could the authors clarify whether the dimensionality and saturation findings remain consistent across these settings?

  4. Have the authors explored persona-conditioned agents in their strategic-game experiments?

2 comments

We thank the reviewer for detailed feedback.

Difference with model homogeneity and correlated error work.

We thank the reviewer for suggesting additional related work. The suggested works [1,2,3] evaluated whether models are similar (e.g., when answer open-ended questions, playing 2x2 games, agreeing on the same incorrect answers). This paper aims to address how many independent directions the differences between models span, and whether that count keeps growing as we add models. This is different than evaluating similarity among models. Suppose all models answer Stag Hunt in nearly the same way, so they all look highly similar. However, if each model adds its own consideration (one mentions reputation, another the chance that the partner makes a mistake, another fairness), then every new model adds something new. If instead models keep adding the same few considerations, new models add nothing. A similarity score can look the same in both cases. We count the distinct kinds of difference using the same embeddings, and find the count stops growing. We will clarify this distinction in our revision.

Does adding diverse models help the team.

We thank the reviewer for suggesting more practical tests. In Section 5, we test multi-agent discussion with different choices of models. A team of three GPT 5.2 copies and two teams of models from different providers both end with no clear gain in strategic score.

Does geometry reflect actual strategic behavior.

We perform additional experiments to evaluate (1) do models that are far apart in embedding space act differently, and (2) does the saturation hold for actions alone, with no text?

For question (1), we compare the distance between two models' average embeddings with:

  • Action disagreement: the chance that answers from models pick different actions.
  • Error-rate gap: the difference between the two models' error rates, where a model's error rate is the share of its answers that miss the target. For example, if one model misses in 30% of its Stag Hunt answers and another in 80%, their gap is 0.5. Our results show that as models get farther apart in embedding space, they disagree more often in their actions and differ more in how often they err.

To answer question (2), we show a similar saturation behavior when only actions are used for embedding in Appendix H. Combining these results together, the geometry is grounded in strategic choices. Adding models bring progressively less new behaviors. We will add these results to our revision.

Ablation on temperature and prompts.

We thank the reviewer for suggesting this experiment. For temperature ablation, we evaluate this using six models from six families that accept temperature setting (Qwen 3.8 Max, GLM-5.2, DeepSeek V3, Gemini 2.5 Flash, Llama 4 Maverick, Kimi K2). We vary temperature as 0.4, 0.7, and 1.0. Our results are reported below:

TemperatureBalanced d90d_{90} (n = 1 →\rightarrow 6)Marginal slope Δd90/Δn\Delta d_{90}/{\Delta n}Model-means d90d_{90}
0.413.2 → 41.07.8 → 3.54
0.714.2 → 44.08.6 → 4.34
1.014.8 → 45.08.7 → 4.04

The results show that the marginal slope falls with every added model and the saturation still holds with varying temperature.

We consider four prompt styles: our prompt, no system prompt, "think step by step", and a one-sentence answer requirement. We evaluate on five models: Claude Opus 5, GPT-5.5, GPT-5.6, Qwen 3.8 Max, GLM-5.2.

PromptBalanced d90d_{90}, n = 1 →\rightarrow 5Marginal slope Δd90/Δn\Delta d_{90}/{\Delta n}Model-means $d_{90}
Our prompt15.0 → 41.09.0 → 4.43
No system prompt14.6 → 31.06.6 → 1.82
Zero-shot chain of thought15.6 → 44.010.1 → 4.83
Decision only9.0 → 17.04.3 → 0.43

We observe that saturation still holds under every prompt, but the level depends on the prompt. Combining the results together, we note that saturating shape is robust to temperature and prompt. We will add this ablation into our revision.

Persona-conditioned agents.

We follow Social Agents and consider 10 demographic/psychographic profiles (age, country, occupation, risk attitude, trust), prepended to system prompt. We evaluate d90d_{90} of response embedding and action-only representation (see Appendix H) and strategic score (Section 5) using five models. We show the results (without personas →\rightarrow with personas) below:

ModelEmbedding d90Action-only d90Strategic score
Claude Opus 518 → 254 → 51.00 → 0.77
GPT-5.511 → 164 → 60.67 → 0.60
GPT-5.613 → 174 → 50.53 → 0.60
Qwen 3.8 Max15 → 225 → 50.53 → 0.57
GLM-5.218 → 245 → 50.80 → 0.73

We observe that personas do not remove saturation. With personas balanced d90d_{90} grows from 20.8 at one model to 58.0 at five. The marginal gain still falls with each added model. Therefore, our scaling result holds with and without personas. We will add this discussion in revision.

Reviewer_qSkMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper runs 50 models on a set of strategic games and measures the diversity of the model outputs. This is done by embedding model outputs and looking at various metrics (PCA dimension, EVR). They find that these metrics have diminishing returns as more models are added. They run the same analysis to compare averaging over (model, game response) pairs vs averaging over models. They also compare different response aggregation methods (majority voting, multi agent discussion, cross provider critique).

Critical strengths

The setting of strategic games is interesting and the paper evaluates a wide set of models and metrics.

Critical weaknesses

The paper uses some jargon that is difficult to parse. For example, what exactly does "geometric monoculture" mean? What does it mean that "the bounded nonlinear core shows that strategic behavior occupies a compact geometry"? I'm able to partially understand these by reading the paper carefully, but wish the high-level ideas were more directly and clearly stated.

The paper's main results show how effective dimensionality grows sublinearly with the number of models. This is consistent with previous literature that has already established that LLMs exhibit homogenous outputs (Wenger and Kenett, 2025, https://arxiv.org/pdf/2501.19361) and correlated errors (Kim et al., 2025, https://arxiv.org/pdf/2506.07962). While the paper does extend these results empirically to the strategic games setting, I'm not convinced that it provides new insights beyond the established work. I'm left wishing for more analysis that would be specific to the strategic games setting (e.g. do LLMs anticipate how other actors will behave in a similar way? does this help to explain the results? do LLMs follow best response dynamics? what happens to the diversity of LLM outputs if you perturb incentives?). Ballestero et al., 2026 (https://arxiv.org/pdf/2604.09502) have studied strategic algorithmic monoculture in a similar setting, and it would be helpful to situate this paper's contributions relative to the existing work.

Finally, the paper frames the fact that LLMs may have similar responses to strategic games as monoculture. But many of the games in the suite have equilibria that can be theoretically characterized. Are the LLM responses aligning with the equilibrium strategy? If so, this should change the interpretation and implications of these empirical results.

Missing citation on monoculture: Kleinberg and Raghavan, 2021 (https://arxiv.org/pdf/2101.05853).

Questions for authors

A large majority of the games in the suite have discrete numerical actions. Does the method embed the LLM's chosen action or the entire LLM response? If this is the LLM's chosen action, I'm not sure how to interpret the "geometry" of these embeddings. It seems that a simpler and clearer alternative would be to look at the empirical distribution over discrete actions (i.e. counts). If this is the LLM's response, then this metric seems to be measuring linguistic diversity of responses. Is that the right thing to measure in this setting?

The paper discusses at length the linear vs nonlinear geometry (PCA vs reconstruction experiment), but does not explicate on why this distinction matters or what it means practically. How should I interpret this result? What does it tell me about diversity or monoculture?

2 comments

We thank the reviewer for the detailed feedback.

Interpretation of "geometric monoculture" and "bounded nonlinear core".

The notion of monoculture is borrowed from algorithmic monoculture (Kleinberg & Raghavan, 2021), where decision-makers use similar algorithms that make correlated decisions. We say the set of models exhibits geometric monoculture when the dimensionality of their joint response space grows with diminishing increments as models are added. The interpretation of geometric monoculture is provided in Line 79-80.

We define the nonlinear core as the nonlinear degrees of freedom needed to reconstruct strategic responses (Line 228-229). The core is said bounded if it does not increase as models are added.

We will clarify all terms in our revision to make them easier to follow.

Comparison with literature.

The reviewer is correct that similarity among LLMs has been established, including pairwise embedding similarity of responses (Jiang et al., 2025, Wenger & Kenett, 2025), agreement on answers (Ballestero et al. 2026), or agreement on errors (Kim et al., 2025). This paper asks a different question: whether adding a model adds new strategic diversity. Suppose all models answer Stag Hunt in nearly the same way, so they all look highly similar. However, if each model adds its own consideration (one mentions reputation, another the chance that the partner makes a mistake, another fairness), then every new model adds something new. If instead models keep adding the same few considerations, new models add nothing. A similarity score can look the same in both cases. We count the distinct kinds of difference using the same embeddings, and find the count stops growing.

Alignment with game equilibrium.

We agree with the reviewer that if models agree on equilibrium, then that is not considered monoculture. Most of the games studied in this paper have no single right answer. Among all games, only 6 games have one best move. Even for these games, models do not all commit to the same equilibria. For example, Claude Opus 5 and GLM-5.2 mostly choose Stag, while GPT-5.5, GPT-5.6 and Qwen 3.8 Max mostly choose Hare for Stag Hunt. Our results show that models can be split into a few groups. Adding a new model does not add a new way to play, Instead, the new model joins one of the existing groups. Across all games, the studied models differ in only about 16 ways.

Missing citation.

We thank the reviewer for pointing our related work. We present a more comprehensive related work discussion in Appendix.

Embed action or entire response.

We embed the entire response including the reasoning and final action. We agree with the reviewer that linguistic diversity may affect the embedding. In Appendix H, we perform embedding with actions only. The results also show that the diversity stabilizes as more models are added to the ecosystem.

Why does the linear vs. nonlinear distinction matter.

Our experiments on linear geometry (PCA) aim to answer the question that as we add models, does the spread of behavior keep growing? The experiments on nonlinear geometry (bottleneck autoencoder) aim to answer the question that is the growth new kinds of behavior or the same kinds arranged along a curve? Our results show that the linear count grows slowly (Table 1). However, this is ambiguous because a rising linear count could be due to (1) new kinds of behavior keep appearing, or (2) the same behaviors lie on a curved surface and new models land at new points on it. Our results show that the linear count rises only from 129 to 137 between 30 and 50 models (Table 1), while the nonlinear plateau stays at 18 for both 30 and 45 models (18–24 across architectures). Our nonlinear results shows that the nonlinear count stays flat, and thus new kinds of behavior do not show up as we add models. So new models land at new points within the same small set of behavioral factors; they do not add new factors. For monoculture, this means models are not identical, but they vary along the same few factors. In practice, adding models gives better coverage of behaviors the ecosystem already has, not behaviors it lacks, which matches our aggregation results.

Thank you for the additional comments and clarifications. However, my main concerns remain and I maintain my original assessment.

Reviewer_mgHnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the setting where an ensemble of LLMs is used to provide an answer in a strategic setting. The main question is whether response quality and diversity improves as the ensemble size increases. Ensembles of varying sizes, up to 50 LLMs from distinct providers, model families, and sizes, are examined on 20 strategic games with varying characteristics. Two forms of variation are measured: response-space variation, and between model variation. The main tools used for the analysis are PCA component-analysis of response embeddings from an autoencoder. The paper finds that (1) )response diversity plateaus as the number of models increases, and (2) there is strong behavior alignment both between model families and within model families.

Critical strengths

The paper analyzes cross-LLM response diversity from an interesting, and to my knowledge, novel angle, as most existing studies on LLM response diversity focus on the single-model scenario. The findings could be practically useful for evolutionary discovery systems such as AlphaEvolve / OpenEvolve, which rely on LLMs to generate novel problem instances.

Critical weaknesses
  • Methodology has unclear connection to motivation: the paper focuses on the strategic behavior of LLMs, yet analyzes embedded textual responses. Why analyze the textual responses rather than the behavior directly? The LLMs are acting within strategic games with discrete action spaces. Thus, one could perform the same type behavioral analysis on the raw action data, which would allow for simplifying the study’s analysis to avoid the use of text embedding models, autoencoders, etc. Beyond just simplifying the procedure, it’s possible that two LLMs behave the same, even if they provide different text justifications.  
  • Related work section is incomplete: the current related work section is only a single paragraph, and does not provide a comprehensive discussion of how the current work relates to/fills gaps in existing literature. For instance, the section claims that the work is novel because it enables ecosystem-level comparisons between models purely from response embedding analysis. However, [1] also characterizes LLM strategic behavior based on behavior, rather than by internal representations. Additionally, the related work investigating LLM diversity also seems relevant.

[1] C. Wang, D. Kasenberg, K. Stachenfeld, P. S. Castro. Discovering Differences in Strategic Behavior between Humans and LLMs. ICML 2026.

Questions for authors

See critical weaknesses.

2 comments

We thank the reviewer for the detailed feedback.

Why analyze textual responses rather than actions directly?

We use embeddings as the primary representation for the following reasons:

  • A move in a game is often more than a discrete action. In Mafia, a move is a public statement followed by a vote on whom to eliminate. In Spy, it is a vote to approve or reject the proposed mission team, followed by a mission action (Success or Sabotage). Embedding the full response, including the reasoning and the final action, keeps the whole move.
  • In multi-agent discussion (Section 5), models influence each other only through what they write. Two models that choose the same action for different reasons send different messages to their teammates.
  • We use open-ended representation to test scaling. Appendix H encodes each final action as a 37-dimensional feature vector. Action-only analysis confirms that variations between models are compact. But it cannot test whether diversity keeps growing as models are added for two reasons.

Related work

We thank the reviewer for pointing out other related literature. Wang et. al investigated how do LLMs differ from humans in strategic behaviors. They use iterated rock-paper-scissors to show that LLMs use richer opponent models than humans. Our paper studies how does strategic diversity among LLMs grow as more models are added. We show that the diversity among models is bounded, and stops growing as models are added. We will include a more comprehensive discussion of literature in our revision.

I thank the authors for their response and for revising the paper to include a more thorough discussion of related work. I consider my second concern resolved, but maintain my first concern on the mismatch between the motivation and the methodology.

The core question of the paper is whether response quality and diversity improves as the ensemble size increases, motivated by modern multi-agent LLM workflows. Given this, it makes sense to me to analyze textual responses. But then why is the analysis conducted on strategic games with simple, integer-based action spaces? Even if Mafia and Spy are textual games, the other 18/20 games are purely strategic, non-textual games. Shouldn't the experiments be conducted on a setting that more closely matches the stated motivation?

In any case, I remain supportive of the paper, and will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.