acceptodds
Withdrawn from ICLR 2027

Emotion-Regulated Internal States for Human-Aligned Social Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models (LLMs) are increasingly used as agents in social simulations to study human behavior across repeated interactions. However, many LLM-based simulations exhibit unstable or unrealistic behavior over time, limiting their ability to model long-term social dynamics. A key reason is that social decisions are often treated as direct responses to situational inputs, overlooking the role of emotion in human decision making. We propose EmoSim, an emotion-regulated framework for LLM-based social simulation. EmoSim models emotion as an explicit internal state represented by low-dimensional emotion vectors that guide agent decisions. Emotional responses are formed through structured social memory, including summaries of past interactions, impressions of other agents, and weak normative regularities, and are grounded in a human dataset of 500 participants reporting emotional reactions across repeated social decisions. Experiments on a third-party punishment task show that EmoSim produces more stable emotional predictions and decision patterns that better align with human behavior.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Social Simulation, Emotional AI, Agents, Third-party Punishment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VbR6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes EmoSim, an LLM-based social simulation framework that places an explicit affective state between social context and action. The agent builds three kinds of social memory (interaction history, impressions of the other player, and normative expectations). It uses them with the current situation to infer valence and arousal via zero-shot prompting. Valence selects one of 8 emotion vectors, calibrated using human valence reports and the model’s hidden states, and arousal scales the strength of the vector injected into the LLM’s hidden states before a punishment decision. The authors collect third-party punishment data from 500 participants over 60 rounds each. With Qwen-2.5-32B-Instruct and Llama-3.1-8B-Instruct, EmoSim outperforms the proposed baselines on emotion and punishment prediction. It also shows better human-distribution alignment on 100 MoralMachine scenarios from SimBench.

Critical strengths
  1. Human emotion data. The dataset could be a valuable resource if released. Many LLM social-simulation works use synthetic or prompt-only emotion conditioning, so grounding emotion in human reports is a meaningful step toward psychologically plausible agents.
  2. Clear separation of emotion formation and decision generation. EmoSim explicitly separates social memory, emotion inference, and emotion-regulated decision, rather than leaving emotion implicit in free-form generation. This makes the mechanism interpretable and supports ablations of individual components.
Critical weaknesses
  1. Baselines are limited and miss the key comparison. Only two dated LLMs are used. The emotion-aware baselines are weak and old. The paper also lacks comparison/discussion of recent emotion-aware or steering methods such as Outraged AI, EAI, Anthropic’s Emotion Concepts, and E-STEER. Since this is a binary repeated-choice prediction problem, the baselines are almost all LLM prompting variants. There is no matched behavioral model trained on the same data, e.g., a mixed-effects logistic model using (X, Y), history, and participant traits. Without one, it is unclear whether the LLM-based architecture is necessary.
  2. The evaluation is limited and confounded by the within-participant split. The main experiment uses a single altruistic punishment task. Transfer is shown only on 100 MoralMachine scenarios. The promised DICES-990 and ISSP results are not available to me, and the paper does not say whether the dataset will be released. The chronological 80/20 split within each participant does not test generalization to unseen people, which a participant-level held-out evaluation would. No confidence intervals or significance tests are reported. The paper motivates the method by behavioral stability, but no direct metric (e.g., temporal consistency, volatility, or emotion-trajectory similarity) supports this.
Questions for authors
  1. Please provide participant details (age, gender, ethnicity distribution), compensation, and the instructions given.
  2. Could the authors report results under a participant-held-out split, where social memory/emotion calibration is built from some participants and evaluated on unseen participants?
  3. Will the human dataset be released?
  4. Please provide the DICES-990 and ISSP transfer results.
Reviewer_PM4vOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a framework called EmoSim, which uses emotional states to guide LLM agents in social decision-making. EmoSim combines different components such as social memory, emotion prediction and emotion vectors together in order to influence an agent's decisions. To be more specific, it first predicts emotional valence and arousal based on the current situation and past interactions, then it uses predicted emotion to modify the model's internal representations.

Within this work, the authors also collect a human dataset from 500 participants. Each participant complete 60 rounds punishment task. They then evaluate EmoSim using Qwen-2.5-32B and Llama-3.1-8B. The final results are compared with direct decision-making and other emotion-based methods.

At the end, EmoSim achieves accuracy 0.7027 and F1 score 0.5782 on Qwen-2.5-32B.

Critical strengths

1). This paper introduced an insightful and interesting approach to modeling emotion in social agents. The paper is using emotional states to guide LLM agents action instead of relying on direct prompting. With Qwen, the F1 score improves from 0.4963 to 0.5782 when the emotion vectors are presented (as shown in table 3) which suggests that modifying internal representations could be useful for social decision-making tasks.

Critical weaknesses

1). The current baselines do not demonstrate the contribution/usefulness of emotion regulation. EmoSim uses previous emotions and punishment decisions (social memory), but direct decision baseline only receives personality desc riptions and current scenario, which is unfair. The paper also are missing simple baselines that use historical decisions. From reported Qwen metrics, approximately 30% of the test decisions involve punishment, so a model that always predicts no punishment would therefore achieve around 70% accuracy, which is quite similar to EmoSim's reported 70.27% result. Improvement should be making sure that is actually coming from emotion regulation.

2). main claims need more supports and some results need clarification. a. Authors repeatedly claims that EmoSim produces more stable emotional responses and decisions, but experiments only report emotion prediction and decision accuracy. b. Figure 3 compares emotion distributions rather than behavior over time. c. The Llama results in Table 1 seem inconsistent, suggesting different punishment rates across methods (45% vs. 30%) on the same test set. The authors should clarify how these metrics were calculated.

Questions for authors

1). Can the authors provide comparison where the baseline can access the same interaction history but without emotion regulation? It would also be useful to include simple baselines based on historical punishment frequencies.

2). The evaluation focuses on prediction accuracy instead of behavioral stability. What evidence supports the stability claim? And does social memory use actual human responses or the agent own predictions during testing?

3). I have a confusion in Table 1, the Llama results seem to imply different punishment rates across methods, although they use the same test split. Could the authors clarify this mismatch?

Reviewer_uLCrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses a third-party punishment experiment with 500 participants × 60 rounds each to build history-based memory, model-inferred valence/arousal, and hidden-state vectors binned by human emotion reports; at inference time it injects the vector plus textual emotion. The authors report decision F1 improvement from 0.3145 (direct decision) to 0.5782 on Qwen 32B, and from 0.5940 to 0.6737 on Llama 8B, along with emotion-prediction ablations and transfer to 100 MoralMachine scenarios.

Critical strengths

1.Real participants reporting both emotion and decisions across repeated rounds provide valuable paired material for studying individual history and the role of emotion. The text-emotion vs. vector-injection ablation (Qwen 32B: full F1 0.5782, no-vector 0.4963, no-text 0.5243) shows that both channels contribute in this experiment (Table 1, Table 3). 2.The chronological 80%/20% split per participant is appropriate for studying repeated interaction; the large degradation in emotion prediction when removing interaction memory suggests that modeling history may be useful (Section 4.1, Table 4).

Critical weaknesses

1.The data protocol and comparisons are insufficient to attribute the gains to an "internal emotion mechanism." Memory directly aggregates past real emotions and punishment frequencies, and the normative prior is estimated from human data, but there is no simple supervised model, lagged-rate/mixed-effects model, or memory model without emotion that uses the same history data under the same budget. The paper also does not state whether memory updates in test rounds use real emotions and behaviors; if so, this is no longer a closed future prediction (Section 3.2, Section 4.1). 2.The "stability/transfer" evidence is insufficient: the main table is a single threshold classification, lacking temporal drift, participant-level uncertainty, and probability calibration. MoralMachine uses only 100 randomly selected scenarios with no error intervals, and the negative SimBench Score in Table 5 lacks a clear definition. The text repeatedly points to a supplementary material that was not attached, limiting verification. The deterministic generation at temperature 0 also needs reconciliation with the Methods section's "sampling from a probability distribution" (Section 3.4, Sections 4.1–4.4).

Questions for authors

1.In the final 12 test rounds, how is interaction memory updated round by round? Do the inputs include the participant's real valence, arousal, or punishment decisions from test rounds? Please provide a closed rolling evaluation using only training history, and compare against simple supervised/text-memory models with the same history information. 2.Please provide participant-grouped bootstrap intervals, cross-participant splits, calibration/temporal-stability curves, and the MoralMachine scoring definition with sampling variance for the 100 scenarios. Clarify whether all ablations match inference count and textual information.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.