- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes EmoSim, an LLM-based social simulation framework that places an explicit affective state between social context and action. The agent builds three kinds of social memory (interaction history, impressions of the other player, and normative expectations). It uses them with the current situation to infer valence and arousal via zero-shot prompting. Valence selects one of 8 emotion vectors, calibrated using human valence reports and the model’s hidden states, and arousal scales the strength of the vector injected into the LLM’s hidden states before a punishment decision. The authors collect third-party punishment data from 500 participants over 60 rounds each. With Qwen-2.5-32B-Instruct and Llama-3.1-8B-Instruct, EmoSim outperforms the proposed baselines on emotion and punishment prediction. It also shows better human-distribution alignment on 100 MoralMachine scenarios from SimBench.
Critical strengths
- Human emotion data. The dataset could be a valuable resource if released. Many LLM social-simulation works use synthetic or prompt-only emotion conditioning, so grounding emotion in human reports is a meaningful step toward psychologically plausible agents.
- Clear separation of emotion formation and decision generation. EmoSim explicitly separates social memory, emotion inference, and emotion-regulated decision, rather than leaving emotion implicit in free-form generation. This makes the mechanism interpretable and supports ablations of individual components.
Critical weaknesses
- Baselines are limited and miss the key comparison. Only two dated LLMs are used. The emotion-aware baselines are weak and old. The paper also lacks comparison/discussion of recent emotion-aware or steering methods such as Outraged AI, EAI, Anthropic’s Emotion Concepts, and E-STEER. Since this is a binary repeated-choice prediction problem, the baselines are almost all LLM prompting variants. There is no matched behavioral model trained on the same data, e.g., a mixed-effects logistic model using (X, Y), history, and participant traits. Without one, it is unclear whether the LLM-based architecture is necessary.
- The evaluation is limited and confounded by the within-participant split. The main experiment uses a single altruistic punishment task. Transfer is shown only on 100 MoralMachine scenarios. The promised DICES-990 and ISSP results are not available to me, and the paper does not say whether the dataset will be released. The chronological 80/20 split within each participant does not test generalization to unseen people, which a participant-level held-out evaluation would. No confidence intervals or significance tests are reported. The paper motivates the method by behavioral stability, but no direct metric (e.g., temporal consistency, volatility, or emotion-trajectory similarity) supports this.
Questions for authors
- Please provide participant details (age, gender, ethnicity distribution), compensation, and the instructions given.
- Could the authors report results under a participant-held-out split, where social memory/emotion calibration is built from some participants and evaluated on unseen participants?
- Will the human dataset be released?
- Please provide the DICES-990 and ISSP transfer results.