acceptodds
Under review as a conference paper at ICLR 2027

Calibrated Co-Evolving Social Simulation for Multi-Step Sentiment Forecasting

Abstract

Sentiment forecasting predicts the evolution of population sentiment distributions as events unfold. Most existing methods treat this as single-step prediction. These methods prompt LLMs to role-play users from historical posts, assume fixed social graphs, and score the prediction at a single target time point. These three assumptions do not hold as the event unfolds. We formulate the task as a calibrated, closed-loop social simulation that couples post generation, network evolution and population-level feedback. The population generates its own future inputs. Each LLM agent represents one user. At each step, active agents respond to their neighbors' posts from the previous step, and the expressed stances drive edge formation and dissolution. History-based participation rates capture heterogeneous participation, while stance-decomposed camp summaries provide population context and preserve within-camp differences through agent-specific injection strengths. To calibrate the resulting dynamics, we combine individual-level and population-level objectives over observed multi-step trajectories, together with homophily-constrained opinion updates. Calibration uses only the observation windows and then freezes the parameters. The simulator runs with no access to future real posts or sentiment distributions. On a 2020 US election Twitter benchmark and a self-built event dataset, our method lowers the macro JSD in the free-forecast window by 85% and 75% over the single-step baseline and raises per-agent accuracy there from 43.0% to 55.5%. The simulated networks are consistent with the real graph on polarization, echo chamber, and opinion assortativity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
llm agents, social simulation, opinion dynamics, sentiment forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_U6mKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies multi-step sentiment forecasting conditional on known events. It combines LLM users, generated feeds, opinion-driven network evolution, and camp-specific sentiment summaries into a simulation loop. With the LLM frozen, five simulation parameters are calibrated using population, individual, and dynamics-consistency objectives, then frozen for forecasting. Evaluation covers calibration and held-out prediction on US election data, followed by transfer to a Ukraine War event without refitting.

Critical strengths
  1. Evaluating both population distributions and individual predictions is sensible, since matching the aggregate distribution does not establish that particular users are predicted correctly. Camp-specific summaries also reflect that different groups can respond differently to the same event.

  2. Transferring parameters from the election to the Ukraine War event without refitting is a useful evaluation. The reported improvements at held-out windows in Table 1 are substantial and make the system worth further investigation.

Critical weaknesses
  1. I found the introduction’s characterization of the research gap too broad, making the approach appear more novel than the comparison establishes. HiSim already calibrates pure ABM parameters against empirical distributions and transfers them to a hybrid simulator [1]. MF-LLM already combines individual–population feedback with empirical alignment [2], while another framework couples text generation, opinion updates, and network rewiring [3]. Directly calibrating the complete frozen-LLM simulator may offer an advance, but its specific contribution relative to these works remains insufficiently explained.

  2. The experiments have not yet convinced me that the gains come from the claimed joint calibration and co-evolution mechanisms. There is only one held-out election step and two Ukraine steps, without separate ablations of individual fidelity and dynamics consistency or a fixed-graph control with matched inputs and calibration budget. The network reference is also the proxy used for initialization, so these results do not establish fidelity to actual tie evolution or sustained free-running forecasts.

[1] Xinyi Mou, Zhongyu Wei, and Xuanjing Huang. Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation. Findings of ACL, pp. 4789–4809, 2024. Paper. [2] Qirui Mi, Mengyue Yang, Xiangning Yu, Zhiyu Zhao, Cheng Deng, Bo An, Haifeng Zhang, Xu Chen, and Jun Wang. MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework. Advances in Neural Information Processing Systems 38, 2025. Paper. [3] Chenhao Gu, Ling Luo, Zainab Razia Zaidi, and Shanika Karunasekera. Language-Grounded Co-evolution of Opinions and Ties: Paired Rewiring with Large Language Model Agents. Proceedings of ICWSM, 20(1), pp. 972–988, 2026. Paper.

Questions for authors
  1. What do the authors consider the main technical advance over [1–3]? In particular, how does joint trajectory calibration substantively differ from existing empirical alignment approaches, and which results directly validate those differences?

  2. Are results available for separately removing individual fidelity, removing dynamics consistency, and freezing only the network structure? Were these variants recalibrated under the same budget? These comparisons would help identify which mechanisms account for the gains.

  3. Appendix E defines turning-point F1 as $ 2|R\cap S|/(|R|+|S|) $. With five windows, there are only three possible interior turning positions, so a single trajectory can yield only (0,0.5,2/3,0.8,1). How were 0.312, 0.342, 0.747, and 0.857 in Table 6 obtained?

Reviewer_8YtnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies multi-step sentiment forecasting using LLM agents whose sentiments, posts, and social ties evolve together. It combines stance-aware population feedback, separate sentiment prediction and post generation stages, and trajectory calibration at both the individual and population levels. Experiments on the 2020 US Election and Ukraine War datasets show gains in forecasting sentiment distributions.

Critical strengths

S1. This paper studies how sentiment changes over several time windows, with users interacting and social ties changing along the way, is a good extension of one-step forecasting.

S2. The proposed framework is make sense that employs network updates, stance-aware population feedback, and trajectory calibration at both the individual and population levels.

S3. The evaluation covers two events (election and war) and includes user, population, and network measures.

Critical weaknesses

W1. The ablation study does not sufficiently isolate the contribution of dynamic network evolution. B5 uses a static followee feed, but it does not directly test the effect of fixing the network topology while keeping the rest of the model unchanged.

W2. Neither dataset contains observed follow relationships. The networks are constructed from proxies, which may differ substantially from actual follower-followee relationships (based on my experience). As a result, the experiments provide limited evidence that the model captures real social network evolution.

W3. The evaluation of multi-step forecasting is still limited. There is only one free-forecast window for the US Election and two for the Ukraine War, making it difficult to assess forecasting stability and error accumulation over longer horizons. The baselines are also mainly traditional opinion dynamics models and a single-step LLM forecaster, with no competitive temporal models designed for multi-step forecasting.

W4. Several evaluation details need clarification. First, simulated sentiment distributions are aggregated over users, while the ground-truth distributions are aggregated over posts, potentially introducing inconsistent weighting. Second, Qwen3-32B is used both for generating most sentiment labels and as the agent model, with human validation conducted on only a small sample. Third, in Table 2(a), the proposed method improves accuracy but does not outperform Single-step MPR in either five-class or three-class Macro-F1.

W5. The Turning-point F1 scores in Table 6 are intersesting but difficult to reconcile with the definition in Appendix E. The metric is defined using the overlap of turning-point sets, and a trajectory with five time windows can have at most three interior turning points. Values such as 0.312, 0.747, and 0.857 therefore cannot be directly obtained from a single pair of trajectories under this definition. The paper should clarify whether these scores are averaged across runs, users, or other units, and provide the exact calculation procedure.

Questions for authors

Q1. What fraction of proxy-network edges comes from retweet matching, and what fraction comes from random connections between users of the same language? How sensitive are the forecasts and network metrics to the choice of initialization?

Q2. How does the method compare with a simple autoregressive model or an extrapolation of past sentiment trajectories? Also, users in the Ukraine War dataset are selected using posting activity across all time windows. Could that retrospective selection affect the conclusions about prospective forecasting?

Q3. Do the macro-level JSD gains hold when the reference sentiment distribution is also aggregated over users? What explains the higher accuracy without a corresponding Macro-F1 gain over Single-step MPR?

Q4. Please give the exact calculation used for Turning-point F1 in Table 6, including any averaging. How does it produce the reported values from the definition in Appendix E?

Reviewer_r4VKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work proposes an LLM based social simulation approach that considers the evolution of social sentiment as a multi-step process incorporating social network evolution. The social network evolution is driven through stance expressed by the agents and its trajectory is calibrated during the calibration window and then evaluated during the forecast window. The experiments show that the proposed approach generates more accurate trajectories compared to the baselines.

Critical strengths
  1. Important problem
  2. Experiments are abundant
Critical weaknesses
  1. Technical depth is poor. Choosing LLM off-the-shelf to design some social experiments may not be appealing to majority of ICLR audience.

  2. Writing is very poor and some terms are used informally. Introduction mentions: “Its opinion is a continuous stance coordinate inferred from its own text at every step, and a bounded-confidence prior over its neighbors enters the objective as a consistency residual rather than governing the opinion directly.” I am unable to understand this sentence.

  • What is stance coordinate?
  • What is consistency residual?
  • What is trust radius (It is mentioned multiple times later without any clarity)? And, yes, it is not a standard term.

Next line follows: “Edges form toward agents of similar opinion and toward well-connected accounts, and they break when two agents diverge beyond the trust radius.” What does this sentence mean?

Few lines later… “The first stage estimates the agent’s sentiment from a camp summary of the macro state, visible only to that camp’s members and injected at an agent-specific strength set by a relevance score.”

  • What is camp summary? What is camp?
  • what is macro state?
  • What is relevance score?

In section 3, we see “Each agent i carries a static persona persona_i and a stance score stance_ score_i \in [−1, 1] resolved from it, where +1 marks strong support for camp_1 and −1 for camp_2 , the two opposing sides of the event under study. “ Kindly proofread.

Then around the end of Intro, we observe another sentence: “On a 2020 US Election Twitter benchmark and a self-built event dataset,”--- what Is "self-built"?

I am unable to move to experiments, though I have one comment about experiments. But, given such poor understanding of the context of the paper, I provide this comment only for due diligence.

  • The best 5- Acc for “Election, Closest” are poor for all methods in Table-1. This means the trajectories are not accurate.
Questions for authors

Check weaknesses and kindly address them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.