acceptodds
Under review as a conference paper at ICLR 2027

Reproducible Social Structure Emerges From Free-Form Interaction Among LLM Agents

Abstract

Multi-agent LLM simulations are increasingly used as models of social systems, but most published findings are anecdotal, i.e., drawn from a single run. It is therefore unclear which observed phenomena are robust findings of the simulated system itself, and which were random events in one particular run. Here, we introduce a repeatable, goal-free, proactive multi-agent framework in which fifty persona-conditioned agents with persistent memory choose whether, with whom, and how to communicate across three social platforms. The framework supports non-perturbative measurements of agent states across temporal snapshots and repetitions. Running the same world ten times from identical initial conditions for a total of 700 hours, we find that most measured network- and agent-level parameters are stable across repetitions. Throughout a run, initially broad communication contracts into sparse sustained dyads in all ten worlds. Importantly, we find that a substantial number of identical dyads re-form across worlds significantly more probably than expected by chance. Static personality similarity largely fails to predict these dyads. Instead, alignment and extremity in the emotions and traits expressed during first contact predict whether a pair later forms a sustained dyad. Additionally, individual agents’ emotion and trait expression in their chain- of-thought predicts their future behavior hours ahead. Finally, swapping either persona prompts or accumulated memories between agents causes recipients to exhibit behavior more similar to donors. Interestingly, persona and memories affect agent behavior to a similar degree. Overall, we show that repeated worlds enable robust conclusions about emergent behavior in agent societies.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM agent societies, emergent social networks, reproducibility, tie formation, activation probes, agent memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_k89tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies social structure in freely interacting LLM agents. The main experiment runs ten worlds with the same fifty personas and evaluates saved agent states separately from ongoing interactions. The authors report recurring sustained dyads, associations between first-contact expression and later dyad formation, and stable network parameters. They also examine how swapping personas and memories changes responses on personality inventories.

Critical strengths
  1. Running the same initial population ten times is a useful design. It helps distinguish patterns that recur across runs from outcomes found in only one trajectory.
  2. Evaluating copies of saved agent states avoids inserting evaluation tasks into ongoing social interactions. Saving personas, memories, and conversation histories also allows the authors to compare agent states at different times and examine the effects of changing their components.
Critical weaknesses
  1. The choice of models and the scope of the cross-model evidence need clarification. Why were no GPT-series models included? The comparisons with Llama, Gemini, and GLM are useful, but mainly concern network statistics and action patterns. It remains unclear whether the main findings on recurring dyads and first-contact expression were also reproduced beyond the Qwen reference worlds. Since Section 3.3 reports substantial differences between models, please explain the model selection and specify which main findings were checked across models, with how many independent worlds.
  2. The chance baseline for recurring dyads does not account for differences between agents. Appendix B.3 assigns every pair the same formation probability of 0.066. However, the same personas are reused across worlds, and some agents may consistently communicate more or receive more exposure than others. These differences could also produce recurring pairs. Figure 2(b) establishes a departure from the uniform-pair baseline, but it does not distinguish partner-specific preferences from stable differences in activity or exposure. Please justify this baseline and clarify how these alternative explanations affect the interpretation.
  3. The conclusion about equal effects of persona and memory is too strong. Section 4 states that they move measured behavior equally, while Appendix B.8 reports a nonsignificant difference between the two swap conditions (p = 0.32). A nonsignificant difference does not establish equivalence. The comparison also uses multiple checkpoints from one world, so it is important to explain how dependence between checkpoints is handled. The results show that both components affect personality-inventory responses, but do not yet establish equal effects on agents’ subsequent social behavior.
Questions for authors
  1. What motivated the model selection, including the absence of GPT-series models? Which main findings were replicated across independent worlds using other models?
  2. Does the interpretation of recurring dyads change after accounting for differences in agents’ activity or exposure?
  3. What evidence supports equal persona and memory effects, and how is dependence between checkpoints handled?
Reviewer_RJxrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether social structures emerging from free-form interactions among LLM agents are reproducible. The authors simulate 50 persona-conditioned agents with persistent memory using Qwen3.5-122B-A10B, repeating the experiment ten times under identical initial conditions. They analyze the formation and recurrence of sustained communication pairs using a four-state Markov chain and a binomial null model. Linear activation probes and regression analyses are used to examine whether agents' expressed emotions and traits predict future relationships and behavior. The authors also assess the stability of network and behavioral statistics across simulation runs and conduct persona-memory transplantation experiments to investigate how these components influence agents' behavioral profiles.

Critical strengths
  1. The paper investigates how initial interactions between LLM agents relate to the formation of persistent relationships and the resulting social network structure. The four-state Markov chain in Fig. 2a provides an interesting abstraction for studying how relationships emerge, persist, or dissolve. I would nevertheless appreciate a stronger connection to concepts from social science and psychology, as I believe a more formal theoretical treatment is still missing (see weaknesses for further details).
  2. The comparisons across persona banks, language models, and platform configurations provide insights into how network properties depend on modeling choices. I also appreciate the counterfactual memory–persona transplantation experiment, which offers an approach to investigating how different components of an agent's state influence its measured behavioral profile.
Critical weaknesses
  1. I am not fully convinced that the related work section is sufficiently comprehensive to justify the five stated contributions. Multi-agent frameworks with persistent memory, autonomous communication, and endogenous social tie formation have already been investigated in prior work ([2] see "suggestions for future work" as one example, my quick online search revealed additional related work). There is also an extensive literature on the role of the persona initialization, and there is a long history of research on relationship formation in social science and psychology (established prior to LLMs, hence placing the contributions within the broader scientific literature would be great). Similarly, repeating stochastic simulations under identical initial conditions is standard experimental practice rather than a methodological contribution by itself. While the analysis of recurring relationships across independent simulation runs is a more distinctive aspect, the paper does not sufficiently establish how the contributions advance beyond existing work.
  2. I have concerns about how reproducibility is evaluated. First, the binomial null model in Fig. 2b assumes that every pair of agents has the same probability of forming a sustained relationship (pˉ\bar p, Appendix B.3). However, agents may differ substantially in how frequently they communicate or form relationships. As a result, the same pairs could repeatedly form relationships simply because certain agents are more socially active, rather than because of any particular compatibility between them. I would therefore expect a comparison against a null model that accounts for individual differences in activity or network degree. In addition, I am not convinced by the stability criterion in Appendix B.7. The authors classify parameters as stable when the average values are sufficiently different from zero relative to their standard errors. However, this does not necessarily mean that these parameters exhibit little variation between simulation runs. Given that reproducibility is a central claim of the paper, I would expect a more direct assessment (e.g. using the coefficient of variation) of the variability of network and behavioral statistics across repetitions.
Questions for authors

Clarification Requests:

  • Dyad recurrence (Fig. 2b, Appendix B.3): Does the observed recurrence of sustained dyads remain significant under a null model that accounts for differences in individual agents' activity or sociability? The current binomial null assumes the same formation probability pˉ\bar p for all agent pairs. Could the authors instead consider pair-specific probabilities pijp_{ij} derived from individual agents' activity levels or degree distributions? This would help distinguish whether recurring dyads reflect compatibility between particular pairs or simply the tendency of certain agents to form more relationships across simulation runs.
  • Parameter stability (Fig. 4a, Appendix B.7): Could the authors provide a more direct assessment of variability across simulation runs, for example through coefficients of variation or prediction intervals? The current stability criterion primarily measures whether parameter means differ from zero rather than whether they remain consistent across repetitions.
  • First-contact prediction (Fig. 3): Does alignment in expressed emotions and traits improve out-of-sample prediction of sustained relationships compared with simpler baselines, such as persona similarity, initial message volume, reciprocity, or individual activity? Could the authors report predictive performance on held-out simulation runs?
  • Persona–memory transplantation (Fig. 5): How do the authors justify the conclusion that persona and memory have similar effects based on a nonsignificant difference between transplantation conditions? Furthermore, do these interventions affect subsequent social interactions, or only responses to the offline personality evaluations?
  • Activation probes (Appendix B.4): Could the authors clarify how the 11 emotion and 241 trait activation directions were constructed for LLaMA-3.1-8B-Instruct and Qwen3-32B, respectively? In particular, how were the emotion probes adapted from the cited work on Claude Sonnet 4.5, and do the 241 trait directions include the Assistant Axis?

Suggestions for Future Work:

  • Please provide your code through an anonymous GitHub repository or as supplementary material in a future submission. As a reviewer, having access to the code would increase my confidence in your obtained results.

  • The literature review and related work discussion are currently incomplete. While the authors mainly cite large-scale agent simulations, related work on the emergence and formation of network structures among LLM agents is not sufficiently covered. A brief literature search reveals several relevant studies that are currently missing, including:

    • [1] Chang, S., Chaszczewicz, A., Wang, E., Josifovska, M., Pierson, E., & Leskovec, J. (2025). LLMs generate structurally realistic social networks but overestimate political homophily. In Proceedings of the International AAAI Conference on Web and Social Media, pp. 341–371.
    • [2] Schneider, P. J., Tian, L., & Rizoiu, M. A. (2025). Learning to make friends: Coaching LLM agents toward emergent social ties. NeurIPS Workshop on Scaling Environments for Agents.

    Regarding persona initialization and reproducible simulation frameworks, I would also point to:

    • [3] Castricato, L., Lile, N., Rafailov, R., Fränken, J. P., & Finn, C. (2025). PERSONA: A reproducible testbed for pluralistic alignment. In Proceedings of the 31st International Conference on Computational Linguistics, pp. 11348–11368.

    This also raises questions about the novelty of the stated contributions. To the best of my knowledge, not all five contributions represent distinct advances over existing work. In particular, aspects of Contributions 1 and 2 appear to overlap with established research (see [1–2]). Rather than emphasizing contributions that have already been investigated, I would encourage the authors to focus more strongly on their analysis of relationship formation using the four-state Markov chain.

  • Overall, I found the paper difficult to fully grasp without consulting the appendix in detail. I believe that restructuring certain sections, specifying the underlying assumptions more clearly, and focusing on a smaller set of empirical research questions would improve the presentation. In particular, the explanation and justification of the binomial null model in Fig. 2b and its implications for the main findings could be strengthened. Similarly, the motivation and assumptions underlying the four-state Markov model in Fig. 2a could be explained more clearly.

Reviewer_KchvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper allows the same group of intelligent agents with memory and personas to repeatedly engage in free social interaction. It finds that some fixed pairings and network structures recur, and that the expressive features in the initial communication can predict subsequent interactions.

Critical strengths

It studies the question of whether a single story can be repeated in agent-based social research by repeatedly running the complete society.

Critical weaknesses

The core conclusion of this paper, that"the discovery that certain fixed pairings and network structures recur," fails to elaborate on its potential significance and impact, thus limiting the paper's influence. Secondly, in the experimental section, reproducibility primarily demonstrates the stability of this fixed setup, but its generalization scope remains limited. The core experiment involved repeating the same model ten times with the same batch of 50 personas; although other models and personas were extended, the main statistical evidence still came from the reference world. In particular, these agents had no external tasks, and relationship contraction may rely on this "nothing to do" environment, making it unsuitable for direct generalization to work teams or competitive systems.

The authors assumed that all pairings had the same probability of forming lasting relationships, uniformly set at 6.6%. However, agent activity, attention levels, and exposure opportunities may differ; repeated pairings with the same batch of active agents could naturally exceed this uniform baseline.

The evidence that "memory and personas have equally significant effects" is insufficient. The p=0.32 between the two transplantation conditions only indicates no detected difference, not that the effects are equal.

Questions for authors

see weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.