- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies multi-step sentiment forecasting conditional on known events. It combines LLM users, generated feeds, opinion-driven network evolution, and camp-specific sentiment summaries into a simulation loop. With the LLM frozen, five simulation parameters are calibrated using population, individual, and dynamics-consistency objectives, then frozen for forecasting. Evaluation covers calibration and held-out prediction on US election data, followed by transfer to a Ukraine War event without refitting.
Critical strengths
-
Evaluating both population distributions and individual predictions is sensible, since matching the aggregate distribution does not establish that particular users are predicted correctly. Camp-specific summaries also reflect that different groups can respond differently to the same event.
-
Transferring parameters from the election to the Ukraine War event without refitting is a useful evaluation. The reported improvements at held-out windows in Table 1 are substantial and make the system worth further investigation.
Critical weaknesses
-
I found the introduction’s characterization of the research gap too broad, making the approach appear more novel than the comparison establishes. HiSim already calibrates pure ABM parameters against empirical distributions and transfers them to a hybrid simulator [1]. MF-LLM already combines individual–population feedback with empirical alignment [2], while another framework couples text generation, opinion updates, and network rewiring [3]. Directly calibrating the complete frozen-LLM simulator may offer an advance, but its specific contribution relative to these works remains insufficiently explained.
-
The experiments have not yet convinced me that the gains come from the claimed joint calibration and co-evolution mechanisms. There is only one held-out election step and two Ukraine steps, without separate ablations of individual fidelity and dynamics consistency or a fixed-graph control with matched inputs and calibration budget. The network reference is also the proxy used for initialization, so these results do not establish fidelity to actual tie evolution or sustained free-running forecasts.
[1] Xinyi Mou, Zhongyu Wei, and Xuanjing Huang. Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation. Findings of ACL, pp. 4789–4809, 2024. Paper. [2] Qirui Mi, Mengyue Yang, Xiangning Yu, Zhiyu Zhao, Cheng Deng, Bo An, Haifeng Zhang, Xu Chen, and Jun Wang. MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework. Advances in Neural Information Processing Systems 38, 2025. Paper. [3] Chenhao Gu, Ling Luo, Zainab Razia Zaidi, and Shanika Karunasekera. Language-Grounded Co-evolution of Opinions and Ties: Paired Rewiring with Large Language Model Agents. Proceedings of ICWSM, 20(1), pp. 972–988, 2026. Paper.
Questions for authors
-
What do the authors consider the main technical advance over [1–3]? In particular, how does joint trajectory calibration substantively differ from existing empirical alignment approaches, and which results directly validate those differences?
-
Are results available for separately removing individual fidelity, removing dynamics consistency, and freezing only the network structure? Were these variants recalibrated under the same budget? These comparisons would help identify which mechanisms account for the gains.
-
Appendix E defines turning-point F1 as $ 2|R\cap S|/(|R|+|S|) $. With five windows, there are only three possible interior turning positions, so a single trajectory can yield only (0,0.5,2/3,0.8,1). How were 0.312, 0.342, 0.747, and 0.857 in Table 6 obtained?