- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies social structure in freely interacting LLM agents. The main experiment runs ten worlds with the same fifty personas and evaluates saved agent states separately from ongoing interactions. The authors report recurring sustained dyads, associations between first-contact expression and later dyad formation, and stable network parameters. They also examine how swapping personas and memories changes responses on personality inventories.
Critical strengths
- Running the same initial population ten times is a useful design. It helps distinguish patterns that recur across runs from outcomes found in only one trajectory.
- Evaluating copies of saved agent states avoids inserting evaluation tasks into ongoing social interactions. Saving personas, memories, and conversation histories also allows the authors to compare agent states at different times and examine the effects of changing their components.
Critical weaknesses
- The choice of models and the scope of the cross-model evidence need clarification. Why were no GPT-series models included? The comparisons with Llama, Gemini, and GLM are useful, but mainly concern network statistics and action patterns. It remains unclear whether the main findings on recurring dyads and first-contact expression were also reproduced beyond the Qwen reference worlds. Since Section 3.3 reports substantial differences between models, please explain the model selection and specify which main findings were checked across models, with how many independent worlds.
- The chance baseline for recurring dyads does not account for differences between agents. Appendix B.3 assigns every pair the same formation probability of 0.066. However, the same personas are reused across worlds, and some agents may consistently communicate more or receive more exposure than others. These differences could also produce recurring pairs. Figure 2(b) establishes a departure from the uniform-pair baseline, but it does not distinguish partner-specific preferences from stable differences in activity or exposure. Please justify this baseline and clarify how these alternative explanations affect the interpretation.
- The conclusion about equal effects of persona and memory is too strong. Section 4 states that they move measured behavior equally, while Appendix B.8 reports a nonsignificant difference between the two swap conditions (p = 0.32). A nonsignificant difference does not establish equivalence. The comparison also uses multiple checkpoints from one world, so it is important to explain how dependence between checkpoints is handled. The results show that both components affect personality-inventory responses, but do not yet establish equal effects on agents’ subsequent social behavior.
Questions for authors
- What motivated the model selection, including the absence of GPT-series models? Which main findings were replicated across independent worlds using other models?
- Does the interpretation of recurring dyads change after accounting for differences in agents’ activity or exposure?
- What evidence supports equal persona and memory effects, and how is dependence between checkpoints handled?