- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces how to identify and classify the collective behavior in different LLM societies through order parameters and a measurement framework. The authors discuss 3 main LLM societies and claim to find self-organisation, information-dependent relaxation shapes, and population-level pathologies despite individual safety tuning.
Critical strengths
The paper offers a model-agnostic way to quantify self-organisation in LLM agent populations, which is a relevant problem that has received little study. Bringing order parameters and null models from complex systems theory into this setting is a reasonable direction, and the inclusion of negative cases (GovSim, ChatEval) is appreciated.
Critical weaknesses
- The framework is unified only in name, and the results are inconsistent. The three systems differ in time units, in the size and nature of their activity spaces, and even in the direction in which Φ moves, so Φ trajectories are not directly comparable across them. Rogue’s second metric (bad reach) is a ratio with its own null rather than “the same operation under a different partition”. In addition, the model-selection outcomes reported in the main text are not always consistent with the appendix tables under the paper’s own criteria.
- Section 3 is conceptually inconsistent. Φ is first introduced as happiness, which increases over time, and is then redefined as entropy, which decreases. Schelling is presented as an example of a phase transition, but “Schelling-style relaxation” is then used as the contrast to phase-transition-like dynamics. “Phase transition” is also redefined as an S-shaped curve in time rather than as behaviour with respect to a control parameter, so the core argument is hard to follow.
- The presentation of results is insufficient. The main text contains only two figures, both on Schelling. The Moltbook trajectory, which is the key evidence for the paper’s main claim, is not plotted anywhere, and no figure shows fitted curves or residuals. Most conclusions rest on prose reports of metrics like R2, ΔAIC and β without systematic organization.
- The introduction is loosely connected to the contribution. The motivation concerns detecting coordinated behaviour in deployed systems, but the paper contains no detection or monitoring experiment, and its fitting procedure is retrospective. The framework itself is introduced in a single sentence. Broader claims, such as environmental information shaping the relaxation regime, rest on a handful of heterogeneous systems with no controlled manipulation.
Questions for authors
The authors are supposed to clarify all the inconsistencies mentioned in the weaknesses and supply necessary additional results and materials to support their claims in the paper.