acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Social Simulation in Open-Population Systems: From Persistent Agents to Communication Processes

Abstract

Large language models (LLMs) have enabled social simulation with persistent agents, where fixed computational individuals maintain internal states and interact over time. However, large-scale **social networks** are inherently **open-population**: potential participants are not fixed in advance, users enter and leave dynamically, and only a sparse subset becomes observable through communication at any given time. In such systems, persistent account identities do not provide continuously observable individual-state trajectories, making recursively maintained psychological states insufficiently grounded in empirical observations. We therefore shift the simulation target from persistent individual trajectories to the observable evolution of social communication, modeling who participates, what they communicate, and how these communications shape subsequent system dynamics. We instantiate this formulation with participant activation from a global user pool, collective state transition, and an LLM policy for communication generation, while explicitly separating environmental inputs from endogenous system evolution. Across five Qwen backbones, our framework reduces average KL divergence by **71.2%** over Direct LLM and **35.0%** over MF-LLM, with boundary and counterfactual experiments further validating the proposed modeling boundaries. Our results establish a principled formulation for faithful long-horizon simulation of open-population social systems.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Social Simulation, Large Language Models, Open-Population Social Systems, Social Networks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kzJkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that LLM social simulators built on persistent agents are ill-posed for open online platforms, where the set of participants changes over time and individuals are observed only when they post. It reframes simulation as recursive prediction of the observable communication history and proposes OpenSim: a communication-volume forecaster with participant activation from a user pool, a mean-field state transition (following M2Sim), and an LLM communication policy trained with a future-trajectory critic. On one Chinese social-media corpus (59 event units, 7 for test) and five Qwen backbones, OpenSim reports lower KL, Wasserstein and DTW than Direct LLM, Social Retrieval and MF-LLM. Intervention, boundary and ablation experiments are also reported.

Critical strengths
  1. Treating participation as a prediction target, instead of supplying the realised participant sequence, exposes an information leak in current mean-field simulators. The boundary experiment (Table 3) measures how much that leak helps (about 35% lower KL). If this holds up, it changes how such simulators should be evaluated.
Critical weaknesses
  1. The headline comparison does not support the claim of simulation without future participant information. The main OpenSim variant samples from an event-specific pool "constructed from the corresponding event's observed participants" (App. D.2), that is, from the people who actually took part, including in the test horizon. This leaks less than supplying the ordered sequence, but it is still future information. With the event-agnostic SocioVerse pool (OpenSim†), the gain over MF-LLM largely disappears, and on two of five backbones it reverses (KL 2.08 vs 1.25 on Qwen2-1.5B; 1.57 vs 1.47 on Qwen2.5-1.5B). The 71.2% and 35.0% improvements in the abstract are therefore measured on the leaky variant.

  2. The participation component is the paper's main conceptual addition, yet it is neither specified nor evaluated. The volume forecaster fωf_\omega is never described. Table 5 lists participant sampling as uniform, so the activation likelihood qψq_\psi appears not to be learned, and App. D.2 states that stepwise communication counts are held fixed in the pool comparison. No experiment reports how accurately volume or participant identity is predicted. The evaluation is also narrow: one Chinese corpus with 7 test units (5 topics), Qwen models only, a single training seed and a single rollout for the main results, and no variance or confidence intervals. The paper does not say whether the baselines are fine-tuned on the same 44 training units as OpenSim. M2Sim, the closest prior method, appears only as a boundary setting in Table 3, not as a baseline in Table 1.

Questions for authors
  1. Please report Table 1 with the event-agnostic pool as the main setting, or explain why sampling from the event's own observed participants does not leak future information.
  2. How is fωf_\omega implemented and trained? At test time, is N^t\hat{N}_t used, or is it replaced by observed counts? How accurate is it? Is qψq_\psi learned, or uniform as Table 5 suggests?
  3. Are Direct LLM, Social Retrieval and MF-LLM fine-tuned on the training units? Can you add M2Sim as a baseline in Table 1?
  4. Can you report variance across training seeds and rollouts, with confidence intervals, for Tables 1 to 3?
  5. (Suggestion) Results on a second dataset, or with a non-Qwen model family, would make the generality claim much more convincing.
Reviewer_TQp3OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that persistent-agent social simulators are ill-suited to open-platform events, where participation is sparse and intermittent and individual states between posts are unobservable. It instead takes the system-visible communication history as the recursive object of simulation, drawing on Luhmann's social systems theory. The proposed OpenSim combines participant activation from a global user pool, a mean-field state transition adopted from M2Sim, and a future-constrained communication policy. Experiments use an intervention-annotated Chinese social-media dataset and five Qwen backbones, with counterfactual intervention, boundary-controlled and ablation studies.

Critical strengths

1. Identifies a real and overlooked evaluation leakage. MF-LLM, M2Sim and related simulators condition their rollouts on the realized future participant sequence. This point is useful for evaluation practice in this area and is the most solid contribution of the paper.

2. Clear problem formulation. The operational system/environment boundary (variables generated by the simulator and recursively incorporated into the history belong to the system; externally supplied variables belong to the environment) is simple and usable.

3. Informative analysis. The fine-grained semantic distributions in Appendix E.2 reveal that the baselines collapse onto "share / promotion / calm" outputs, which is an informative observation.

Critical weaknesses

1. The relationship to M2Sim and the attribution of gains should be clarified. Since the mean-field state transition follows M2Sim and the future-constrained objective builds on M2Sim’s long-horizon consistency objective, the new components are mainly the response-sensitive critic, the hinge sensitivity loss and inference-time candidate selection. It is therefore unclear how much of the improvement comes from the reformulation itself versus the inherited components. The paper should state its relationship to M2Sim more explicitly and design experiments that isolate the effect of the formulation.

2. Theoretical grounding and conceptual distinction remain unclear. The theoretical positioning of OpenSim raises two concerns:

  • Relationship to Luhmann's theory. The connection appears largely analogical rather than methodological. In particular, placing official communications in the environment conflicts with Luhmann's view that social systems consist of communications. Moreover, OpenSim models aggregate attitudes rather than connections between communications, leaving its theoretical grounding unclear.
  • Distinction from persistent-agent approaches. If users can access their own earlier posts, OpenSim may function similarly to memory-based persistent agents. The distinction appears to rely mainly on whether an explicit latent state is maintained, which is insufficiently sharp given that persistent-agent approaches need not assume psychologically grounded states either.

3. The central claim of not requiring future participation information is not established experimentally.

  • The event-specific participant pool itself carries future information. The OpenSim pool is constructed from the participants observed in the corresponding event. The model does not know the order of participation, but it knows the set of users who eventually participate over the whole event, together with their features, which is equally unavailable at genuine inference time. The claim of genuine forward simulation is therefore overstated.
  • Participation modeling is effectively not learned. Sec 3.1 introduces f_ω and q_ψ, but D.2 states that both pools are sampled uniformly and that stepwise communication counts are held fixed in the pool comparison. Please clarify:
    • Is q_ψ simply uniform in the main experiments?
    • Is N̂_t predicted, or taken from the ground truth? If the latter, this is itself a form of future leakage.
  • Participation prediction is never evaluated. The theoretical argument (B.2) stresses that D_part must be tested, yet none of the reported metrics measures participation itself: there is no volume error, no activation accuracy, and no analysis of re-entry patterns. There is a clear gap between the motivation and the evaluation.

4. Incomplete baselines.

  • The most directly relevant baseline, M2Sim, is missing. OpenSim inherits its state transition and consistency objective from M2Sim, which is the most closely related method and should be included in Table 1.
  • Training-based baselines are missing. Direct LLM and Social Retrieval have NLL values of 13–25, suggesting prompting, whereas OpenSim is trained with LoRA SFT and joint optimization. The 71.2% reduction over Direct LLM largely reflects trained versus untrained models.

5. The actual contribution of the LLM to macro dynamics is unclear. At inference, the critic selects, among four candidates, the one most consistent with the future mean field predicted by the state model, and the main metrics (KL / Wasserstein / DTW over attitude distributions) are themselves mean-field-level quantities. The improvement in macro metrics may therefore be driven mainly by the state model, with the LLM merely conforming to it. Please add:

  • results of the state model alone, outputting attitude trajectories directly without LLM generation;
  • classical time-series or Hawkes-type baselines for volume and attitude shares.

6. A single dataset; generalization is not validated.

  • Very few effective test samples. The test set contains only 7 units from 5 underlying topics, and all results come from a single run without variance or confidence intervals.
  • Possible topic leakage. If the split is by unit rather than by topic, different intervention contexts of the same event may appear in both training and test sets. Please specify the split; if it is by unit, please report results with a topic-level split.
  • No generalization axis is tested. One platform, one language, one time granularity (days) and one action space (comment / share only). There are no cross-platform, cross-lingual, cross-event-type or temporal extrapolation experiments, and the "consistent across backbones" claim is limited to a single model family (Qwen).
Questions for authors

Please refer to the Weaknesses

Reviewer_XfzkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper discusses a limitation of LLM-based social simulation. Existing systems are built around persistent agents, and they do not fit open online platforms well. Based on this observation, the paper reformulates open online social simulation, shifting the simulation target from persistent individual trajectories to the observable evolution of social communication, modeling who participates, what they communicate, and how these communications shape subsequent system dynamics. By evaluating across five Qwen backbones, the resulting framework improves long-horizon trajectory fidelity without access to the realized future participant sequence.

While the observation stated in the paper is interesting, there are several major flaws in the paper, which are stated in "Critical Weaknesses".

Critical strengths

S1. The conceptual framing is thoughtful and well motivated. It centers around the observation that LLM-based social simulators built around persistent agents fit open online platforms poorly. I agree with this argument.

S2. Separating declared environmental inputs from endogenous dynamics is reasonable.

S3. Ablation studies are extensive.

Critical weaknesses

W1. For the two variants of the proposed framework (OpenSim and OpenSim$\dagger$), (1) it is unclear how the experimental design guarantees that OpenSim works without access to the realized future participant sequence; (2) while OpenSim$\dagger$ uses an external participant pool from SocioVerse, its performance is less competitive, sometimes worse than MF-LLM.

W2. The evaluation only covers models in the Qwen family. Some models are outdated, e.g., Qwen 2 and 2.5. For Qwen 3, only 1.7B was evaluated. It is unknown if the results can transfer to other models, especially more advanced models from multiple families.

W3. The paper states that the proposed framework follows M^2Sim, and M^2Sim also appears in the benchmark comparison (Table 4). However, the experiments did not include it as a baseline.

W4. The scale of the experiments is unclear. For example, the paper does not report the size of N^t\hat{N}_t.

Questions for authors

Q1. The experimental setup shows that "OpenSim uses an event-specific participant pool constructed from the corresponding event’s observed participants". How do you exclude future participants from the set of users who participate during the event? This question is raised because it is concerned with the claim that the proposed framework works without access to the realized future participant sequence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.