acceptodds
Under review as a conference paper at ICLR 2027

Too Categorical to be Human: Emotion Concepts in LLMs and Humans

Abstract

Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. With evidence of emotional stimuli shaping high-stakes behavior in Large Language Models (LLMs), there is increasing interest in how models represent emotion concepts internally. Mechanistic accounts of these representations, however, cannot be compared directly against humans: emotion processing in humans is highly distributed and yields no equivalent neural representation. To understand whether LLMs internalize emotion concepts in a way similar to humans, we propose characterizing the abstract concept of an emotion using external behavioral signatures, which we term behavioral representations. Using the theory of cognitive appraisals, which enables representing emotional situations along interpretable evaluative dimensions (e.g., how pleasant or effortful a situation is), we create a benchmark dataset of emotional scenarios spanning 15 emotion categories. We elicit behavioral representations of emotion concepts from LLMs and humans using our benchmark, and study their structural similarity. We find that LLMs represent emotion concepts more categorically, homogeneously, and determinately than humans, representing a single emotion concept with less internal diversity, and place different emotions further apart. Models from different families converge with one another on how they represent a given emotion significantly more than groups of humans do. The categorical structure of representations in LLMs is further robust to contextual variation, including with different task framing and demographic personas. Analyzing model checkpoints across different training stages, we also find that the discretized nature of representations appears after the mid-training stage itself and is unaffected by different post-training strategies. Through our results, we highlight a key difference in how LLMs behaviorally represent emotion concepts, curbing the subjectivity inherent to the human experience of emotions. More broadly, we argue that evaluating LLMs in subjective domains such as emotions requires moving beyond static prediction to assess the structure of what models represent.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
emotion understanding in LLMs, human-AI alignment, emotion structure, affective computing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_KquaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper aims to characterize emotion concepts through behavioral representations for LLMs that can be compared with human better directly. The authors build the CoRE benchmark on cognitive appraisal theory. Based on the representation and the benchmark, the paper finds that LLMs and humans share a broadly similar overall structure, but LLMs organize emotion concepts in a way that is more categorical (different emotions lie further apart and instances of the same emotion cluster more tightly), more determinate (emotions can be distinguished more accurately from the ratings alone), and more convergent (different models agree with one another more than different groups of humans do). These properties persist across changes in prompt context and are already present before post-training. The authors conclude that LLMs lack the context-dependent variability that characterizes human emotion.

Critical strengths
  1. The paper sets out to develop a characterization of emotion concepts that allows direct comparison between humans and LLMs, which is a valuable goal. This characterization is grounded in classic cognitive appraisal theory.

  2. Building on this representation, the paper conducts a systematic analysis from multiple perspectives. Its core findings — that in LLMs, different emotions lie further apart, instances of the same emotion cluster more tightly, and emotions are easier to distinguish — are supported by multiple metrics, including Wasserstein distance, logistic regression, and SynthTree, all of which converge on the paper’s central result regarding structural alignment. The experiments are fairly thorough.

Critical weaknesses

W1. Mismatch between the level of the conclusions and the methodology.

The abstract motivates the study by asking “whether LLMs internalize emotion concepts in a way similar to humans,” and the title and conclusions state that LLMs represent emotion concepts more categorically. However, This paper does not attempt to locate the directions or subspaces in which models internally encode emotion information. Instead, it compares the explicit ratings that models and humans give on a "predefined questionnaire". In my view, alignment in external behavior and that in internal representation are questions at two different levels. The current evidence therefore supports only the claim that response patterns on the appraisal questionnaire differ; it is not sufficient to establish that the internalized emotion concepts differ. I suggest that the authors narrow their claim.

W2. Potential circularity in dataset construction.

Source bias: most of the scenarios in CoRE were generated by GPT-4o expansion (Stage 2 of the four-stage curation pipeline; Section 2, Fig. 6). The only human-written scenarios are the seed scenarios from Stage 1 and the alternative scenarios provided by annotators in Stage 3. Meanwhile, the evaluated models include GPT-family models. I am concerned that this construction may introduce a systematic bias into the evaluation results, since LLMs are effectively being evaluated on LLM-generated material.

Questions for authors

Q1. To what extent do the observed differences between humans and LLMs stem from biases in the dataset itself? I would like the authors to report the proportions of human-written and GPT-4o-generated scenarios among the final 270 scenarios. If possible, I would also like them to repeat the main analyses (separation ratio and classification accuracy) separately on the human-written and GPT-4o-generated subsets, to check whether the conclusions remain consistent with those reported in the main text.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their thoughtful engagement with our paper. We are also glad that the reviewer found our core findings convergent and our experiments thorough. We respond to each of the weaknesses and questions raised below:

(W.1) "..Mismatch between the level of the conclusions and the methodology."

We agree that alignment in external behavior and alignment in internal representations are questions at different levels, and our study addresses the first. The paper states this at several points:

  • The abstract notes that mechanistic accounts "cannot be compared directly against humans", because emotion processing in humans is distributed and yields no equivalent neural representation. -The introduction defines our measurements as behavioral representations, "interpreting them functionally, and not as markers of neural or activation-level representations" (l. 99–101).
  • The related work reviews studies that locate emotion information in layers, neurons, and subspaces of LLMs, and explains that, because human emotion processing yields no internal representations comparable to model activations, we compare the two through self-reports (l. 148–150).
  • Our limitations section (Appendix A.1) further notes that appraisal judgments are one of several ways to operationalize behavioral representations.

We will additionally revise two specific phrases in the abstract and introduction, which ask whether LLMs "internalize" emotion concepts, and can be interpreted as a claim about internal representations. We will state further explicitly in the introduction and conclusion that our findings concern behavioral representations elicited through appraisal judgments, measured at a different level from studies of internal model representations.

(W.2) “Potential circularity in dataset…”

(Q.1) “To what extent do the observed differences….proportions of human-written and GPT-4o-generated scenarios…repeat the main analyses (separation ratio and classification accuracy) separately on the human-written and GPT-4o-generated subsets…”

Within the final scenarios, 94 (~35%) are directly human-authored, across the first two stages of data curation. Additionally, 72% of the scenarios in the final set are rewritten or edited lightly in the final stage of verification by the research team (Stage 4).

We repeat the separation-ratio and classification analyses separately on the purely human-authored scenarios and the remaining scenarios. Classification uses the logistic classifier from the main text, with its accuracy computed separately on each subset.

Human-writtenRemaining
Separation ratio, humans1.211.32
Separation ratio, models (range)1.68 - 2.131.83 - 2.44
Classification accuracy, humans0.400.27
Classification accuracy, models (range)0.60 - 0.800.65 - 0.82

Our main conclusions remain the same in both subsets. Every model has a smaller within-emotion spread than humans (Holm-corrected p < 0.005 in both subsets), a larger between-emotion distance (p < 0.008 in both subsets), and a higher per-emotion F1 (p < 0.03). Retraining the classifier within each subset leads to the same result (humans 0.38 and 0.35; models 0.63-0.81 in both subsets). Humans classify the human-written scenarios slightly better than the generated ones, while models do not show a consistent difference, so the human-model gap narrows slightly on the human-written scenarios but remains large. The observed differences are therefore not explained by the origin of the scenarios. We will add these results to Appendix B.2.

We hope these clarifications and the new results address the main points underlying the initial evaluation, and we would be very grateful if the reviewer would consider updating their score accordingly!

AuthorsOpenReview ↗

Request to Engage

Dear Reviewer,

As the discussion period is coming to an end soon, we were wondering if our rebuttal addresses your current set of concerns. If it does, we would be very grateful if you could consider updating your score accordingly. Otherwise, we are happy to provide further clarifications or answer any other questions if there are any remaining concerns.

Thank you for the detailed response and the additional analyses. My two main concerns have therefore been largely addressed, and I will maintain my score of 3 (weak accept).

One observation from the new results may be worth a brief comment. For human raters, the two metrics move in opposite directions across the subsets: on the human-written scenarios, the separation ratio is lower (1.21 vs. 1.32), yet the classification accuracy is higher (0.40 vs. 0.27). One would usually expect less separable categories to be harder to classify. It would be helpful if the authors could explain this discrepancy, for example whether it reflects the small size of the human-written subset (94 scenarios, about six per emotion), and whether it still holds with a larger set of human-written scenarios.

Either way, I find this an interesting result.

AuthorsOpenReview ↗

Thank you!

Thank you for your response! We are glad the rebuttal largely addressed both of your concerns!

We also thank you for the observation about the diverging metrics. As suggested, the size of the human-written subset plays a central role, and we examined both metrics with the two subsets compared at equal size.

For the separation ratio, when both subsets are drawn down to the same number of scenarios per emotion (200 random draws), the human ratio is 1.21 on the human-written and 1.24 on the generated scenarios (difference −0.03, 95% range −0.11 to 0.04). The lower ratio on the human-written subset therefore reflects its smaller size.

For classification accuracy, we repeated the cross-validation with 100 different random divisions into folds and averaged, for each scenario, how often it is classified correctly. On average then human accuracy is 0.41 on the human-written and 0.31 on the generated scenarios. This difference is at the margin of statistical significance (permutation test, reassigning scenarios between the two subsets within each emotion: p = 0.06), and it does not appear when the classifier is trained separately on each subset with the same number of scenarios per emotion (human accuracy 0.37 and 0.41; difference −0.04, 95% range −0.15 to 0.06). The difference between models and humans, in contrast, holds in every division: every model exceeds humans by at least 0.17 on the human-written and 0.29 on the generated scenarios.

At equal size, the two subsets therefore do not differ in separation for humans, and humans show at most a modest tendency to classify the human-written scenarios more accurately. The difference between models and humans, which is central to our main conclusions, remains large in both subsets. We will include this additional result and interpretation in the appendix. Thank you!

Reviewer_hJgSOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript investigates the structural organization of emotion concepts in LLMs compared to humans using a novel benchmark named CoRE. CoRE contains 270 hypothetically framed scenarios spanning 15 emotion categories, each rated along 17 cognitive appraisal dimensions. Authors collect first-person appraisal ratings from 41 retained US human raters and 8 diverse LLMs. Using multivariate Wasserstein distance metrics, the paper quantifies between-emotion separation and within-emotion spread. Experimental validation reveals that LLMs exhibit a higher separation ratio than humans, achieving higher emotion classification accuracy and greater internal consistency between explicit statements and implicit representations. Additional robustness experiments show that this discretized structure persists across unlabeled prompts, third-person framing, and demographic personas, and emerges after mid-training rather than post-training alignment.

Critical strengths

The manuscript is well-written, logically structured, and clearly presented. Figs. 1 and 6 provide intuitive overviews of the benchmark curation pipeline and evaluation setup. The appendices provide exact prompt templates, statistical derivations, dataset comparisons, and exhaustive experimental details. The manuscript reveals that while LLMs share coarse geometric dimensions with humans, they organize subjective concepts into overly compressed, discrete, and convergent representations. This provides valuable theoretical and practical insights for human-AI interaction, agent simulation, and safety alignment. The paper introduces a compelling framework that moves beyond standard accuracy benchmarks to evaluate the structural topology of abstract concepts in LLMs. Grounding behavioral representations in established cognitive appraisal theory provides an interpretable, 17-dimensional evaluative space that enables direct, functionally comparable human-LLM benchmarking. The experimental validation is thorough. Authors perform noise-matching experiments (injecting human sampling error into LLMs), extrapolate to infinite human rater panels, evaluate four distinct contextual variations, and trace checkpoint emergence across OLMo 3 family training stages.

Critical weaknesses

The scenario-level averaging individual human ratings suppresses real individual variability in how humans appraise emotional situations, contrasting multi-person human averages against single-run deterministic outputs of individual LLMs. Furthermore, the human reference sample is restricted to 41 US-based participants, limiting the universality of the human benchmark.

Operationalizing behavioral representations through Likert scale self-reports in text prompts introduces the risk of measuring LLM prompt-following constraints rather than deep cognitive representations. LLMs are known to exhibit numerical scale clustering, rounding biases, and high consistency when prompted with category names, which may artificially inflate the observed discreteness and explicit-implicit rank correlations compared to human self-reports.

Questions for authors
  1. How do the between-emotion distance, within-emotion spread, and separation ratio change if the human baseline is evaluated using individual participant profiles directly rather than scenario-averaged profiles?
  2. Did the authors evaluate scale-free or comparative elicitation methods (e.g., pairwise scenario comparisons or ordinal rankings) to verify that the high discretization and determinacy in LLMs are not artifacts of Likert scale prompt adherence?
  3. In the OLMO checkpoint analysis, reasoning-heavy mid-training increased discretization. Could the authors clarify whether enabling explicit chain-of-thought reasoning in DeepSeek R1 or Gemini 3.7 Flash further accentuates or mitigates this concept-level discretization?
5 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We thank the reviewer for their thoughtful engagement with our paper. We are also glad that the reviewer thought our paper provides valuable theoretical and practical insights for agent-based simulation and alignment, and also found our experimental framework compelling. We address the weaknesses and questions raised here:

(W.1 (part a)) “The scenario-level averaging individual human ratings suppresses real individual variability…”

(Q.1) “How do the between-emotion distance, within-emotion spread, and separation ratio change if the human baseline is evaluated using individual participant profiles directly…?”

We repeated the main analysis with individual participant profiles in place of scenario averages. For each scenario, we used the unaveraged 17-dimensional rating of one participant, chosen at random from all participants who gave a complete rating of that scenario, and repeated this over 100 random draws. Each human emotion representation is therefore built from exactly as many points as a model's (one per scenario):

Between-emotion distanceWithin-emotion spreadSeparation ratio
Humans (scenario averages)4.653.601.29
Humans (individual participants)4.463.991.12 (1.08 to 1.17 across 100 draws)
Models (range)5.31 to 5.442.15 to 2.801.90 to 2.53

With individual profiles, the human within-emotion spread increases and the separation ratio decreases, so the human representations become even less categorical.

Compared with the human values averaged over the 100 draws, every model has a smaller within-emotion spread on all 15 emotions (Holm-corrected p < 0.001, rank-biserial −1.00) and a larger between-emotion distance (Holm-corrected p < 0.001). Every model's ratio exceeds the human ratio in all 100 random draws.

On the model side, single responses are not a source of bias either: in repeated sampling (details in our response to Reviewer CiL5), independent responses from models agree closely (mean ICC(1,1) = 0.88 to 0.97), and the separation ratio from a single response is close to that from the average of four responses.

(W.1 (part b)) “...the human reference sample is restricted to 41 US-based participants…”

As stated in our limitations (Appendix A.1), the human sample does not support claims about universality across human populations. We, however, do not expect this to change our main conclusions, for two reasons:

  • First, a sample from a single country is likely to be more homogeneous than a global one, because people from different cultural backgrounds appraise the same situations differently. A broader sample would therefore be expected to increase the human within-emotion spread and widen the human-model difference.
  • Second, prompting models with nationality-based personas from five countries leaves their representations as discretized as in the persona-free condition (Section 3.2, Fig. 4; Appendix M). The model side of the comparison is therefore stable across the tested cultural framings, which makes it unlikely that the conclusions would change substantially when models are compared with more diverse human populations.

(W.2) “Operationalizing behavioral representations through Likert scale self-reports…”

(Q.2) “Did the authors evaluate scale-free or comparative elicitation methods…”

To test whether the result depends on how the rating scale is used, we repeated our main analysis using only ordinal information. For each agent and each appraisal dimension, we replaced the ratings across all scenarios with their ranks, which removes which scale points were used and how far apart they are.

The main result is unchanged in direction and significance. The separation ratio is 1.28 for humans and 1.84 to 2.38 for the eight models (1.29 and 1.90 to 2.53 with the original ratings, respectively). Every model has a smaller within-emotion spread than humans on all 15 emotions (Holm-corrected p < 0.001), and a larger between-emotion distance overall (Holm-corrected p < 0.001). The difference between models and humans therefore does not depend on the numeric spacing of the scale.

Further, from our existing results, we see that removing the emotion label from the prompt leaves every model still significantly more discretized than humans (Section 3.2), also when only the scenarios a model interprets as the intended emotion are considered (Appendix K), so the result does not depend on naming the emotion category. Using further different elicitation formats can also be an interesting follow-up, although obtaining behavioral representations in a higher-dimensional space would involve significant costs if built through a pairwise ranking method.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2 - Continued from above)

(Q.3) “In the OLMO checkpoint analysis, reasoning-heavy mid-training ... enabling explicit chain-of-thought reasoning in DeepSeek R1 or Gemini 3.7 Flash further accentuates or mitigates ...”

We thank the reviewer for this interesting suggestion! We ran a chain-of-thought (CoT) condition for Gemini 3.7 Flash and GPT-5.6 Luna (DeepSeek R1, as an older reasoning model, already generates a reasoning chain before every answer). The prompt is unchanged except that the model is asked to write out its step-by-step reasoning about the appraisal question before providing its rating, in a separate field placed before the rating. Every response obtained followed this format. We report the recomputed separation ratios:

ModelWithout CoTWith CoTHumans (same scenarios)
Gemini 3.7 Flash2.532.551.29
GPT-5.6 Luna2.432.291.29

For Gemini 3.7 Flash, the ratio with CoT remains close to the responses in the original condition. For GPT-5.6 Luna, the ratio is slightly lower with CoT, through a larger within-emotion spread (p = 0.02). Even with CoT, both models remain far more categorical than humans: a smaller within-emotion spread on all 15 emotions (p < 0.001) and a larger between-emotion distance (p < 0.001). Explicit reasoning in the prompting stage therefore does not accentuate the discretization, and it does not bring it close to human levels.

We hope these clarifications and the new results address the main points underlying the initial evaluation, and we would be very grateful if the reviewer would consider updating their score accordingly!

AuthorsOpenReview ↗

Request to Engage

Dear Reviewer,

As the discussion period is coming to an end soon, we were wondering if our rebuttal addresses your current set of concerns. If it does, we would be very grateful if you could consider updating your score accordingly. Otherwise, we are happy to provide further clarifications or answer any other questions if there are any remaining concerns.

Reviewer_UZFoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CoRE, a benchmark of emotional scenarios evaluated by humans and eight LLMs along interpretable cognitive appraisal dimensions. It compares variation within emotion categories, separation between categories, agreement across models, and consistency between explicit judgments and scenario-based ratings. The authors report that LLM ratings form more distinct and internally homogeneous emotion categories than human ratings. They examine these findings through alternative classifiers, controls for measurement noise, prompt variations, and checkpoints from the OLMo 3 training pipeline. The main contribution is a benchmark and empirical analysis of differences that average prediction accuracy may overlook.

Critical strengths

The study makes a useful comparison on shared stimuli. Evaluating humans and models on the same scenarios and appraisal dimensions makes the differences interpretable. The focus on how judgments vary across situations within one emotion is valuable for assessing whether models capture more than a typical appraisal profile.

The authors investigate plausible alternative explanations for the main finding. The controls for emotion labels and measurement noise, together with the agreement between distance analyses and classification results, make the difference in category separability reasonably convincing. These checks give the empirical finding more weight than a comparison based on a single metric or prompting condition.

Critical weaknesses

The advance over the closest prior work is narrower than Section 1.1 suggests. Tak and Gratch (2024) [1] already report greater emotion-category separability in GPT-4’s appraisal ratings, while Zhou et al. (2025) [2] examine appraisal distributions, including their variability. CoRE adds a useful comparison of variation across scenarios within an emotion and a broader set of controls. However, much of the evidence extends an established pattern. The paper does not yet make a sufficiently clear case for which additional conclusion substantially changes our understanding. This limits my assessment of its informativeness despite the substantial experimental effort.

[1] Ala N. Tak and Jonathan Gratch. 2024. GPT-4 Emulates Average-Human Emotional Cognition from a Third-Person Perspective. arXiv preprint arXiv:2408.13718. https://arxiv.org/abs/2408.13718

[2] Yuxiang Zhou, Hainiu Xu, Desmond C. Ong, Maria Liakata, Petr Slovak, and Yulan He. 2025. Modeling Subjectivity in Cognitive Appraisal with Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 13811–13833. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-emnlp.744

The statistical validation does not fully establish the reliability of the structural effects. Appendix C.5 evaluates the stability of mean emotion profiles, whereas the main conclusions depend on variation within those categories. In addition, the tests in Appendices F and H use emotion pairs that share categories and underlying ratings, without explaining how this dependence is handled in the inference. The controls for measurement noise are useful, but they do not replace uncertainty estimates that account for participant and scenario sampling. This leaves the precision of the structural comparisons less clear than the reported significance levels suggest.

Questions for authors

Additional analysis using existing data: Could you report uncertainty intervals for the main structural differences that account for sampling across participants and scenarios? A direct measure of within-emotion dispersion, such as the covariance trace, would also help establish whether the compression finding persists beyond the split-half Wasserstein measure.

Clarification: Appendix H states that individual human ratings are pooled, but Table 11 attributes the change in the human correlation from 0.058 in Table 10 to 0.02 to pooling rather than averaging. What aggregation is used in each analysis? What is the reliability of the human explicit dimension-importance judgments, which also affects the interpretation of their low explicit–implicit correspondence?

Additional analysis using existing data: Does the training-stage finding persist when using expected ratings or comparing the full rating distributions? Appendix O uses the most probable rating to construct the geometry. Probability distributions can become substantially sharper without changing this rating, so the current analysis does not establish that post-training leaves the broader behavioral representation unchanged.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We thank the reviewer for their thoughtful engagement with our paper. We are also glad that the reviewer appreciated our experimental setup with shared stimuli and the additional control conditions we studied. We address the weaknesses and questions raised here:

(W.1) “The advance over the closest prior work is narrower….”

Although the cited papers are indeed important in the space of appraisals in LLMs, several important differences exist:

  • Both cited papers measure prediction, while we measure the structure of emotion concepts: In both cited studies, LLMs are required to predict another agent’s emotion or appraisal. Notably, in Tak and Gratch (2024), GPT-4 predicts the appraisal distribution given a scenario alone (without the emotion) or the emotion category given the scenario itself. Thus, they do not capture appraisal-based representations of a given emotion concept, but that of a scenario alone. Zhou et. al. (2025), on the other hand, involve prediction of only appraisal distributions, and do not study any emotion categories at all. Both of the study designs are thus not set up to capture what we study: appraisal-based structural characterization of emotion concepts. Also, in both studies, predicting the appraisal is also treated as the end goal, while we use appraisals as a tool to help characterize behavioral representations of emotions.

  • Neither design compares people and models under the same conditions: Tak and Gratch’s (2024) human data come from an earlier observer study run under its own protocol, and their only first-person appraisals are each author’s ratings of their own event, one person per stimulus. Zhou et al. (2025) use existing corpora of observer annotations and tune model sampling temperature to match the human spread. CoRE is built to enable studying human-alignment, including the same 270 scenarios, the same verbatim appraisal questions, and the same first-person instruction for human participants and LLMs, with several individuals providing self-appraisal judgements for a given scenario.

  • Both studies measure and find different properties: Tak and Gratch (2024) find that GPT-4’s appraisal ratings yield more clusters of the scenario stimuli than people’s (20 against 12) and recover the intended label more accurately, that is, GPT-4 shows a finer-grained separation between the provided scenarios. Showing that GPT-4 distinguishes more categories differs from our main finding that LLMs from multiple families represent emotions more categorically: with less variation across the situations of one emotion and sharper boundaries between emotions. Notably, Tak and Gratch (2024) do not include any within-emotion comparisons to enable capturing the concept-level homogenization we measure. Zhou et. al. (2025) measure disagreement between different people about a single situation (when providing third-person appraisals) without involving emotion categories altogether, which does not allow any analysis or conclusions about any emotion concepts.

Along with these main differences, our study also adds a comparison across eight models from different families, a comparison of explicit and implicit judgments, and an analysis across training stages. We will revise Section 1.1 to state these distinctions.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2 - Continued from above)

(W.2) “The statistical validation does not fully establish…”

(Q.1 (part a)) “.....uncertainty intervals for the main structural differences that account for sampling across participants and scenarios?...”

To account for sampling of participants and scenarios, we repeat our main analyses on 500 random subsets, each containing 80% of the participants and 80% of the scenarios of every emotion, drawn without replacement. In each subset, we rebuild all human data from the selected participants only, use the model ratings for the same scenarios, and recompute every quantity unchanged (the models' explicit judgements in Appendix H involve no participants or scenarios and stay fixed).

Because the subsets resample participants and scenarios and not emotion pairs, a subset that changes the scenarios of one emotion changes every pair involving that emotion, so the intervals reflect the dependence between pairs instead of assuming independence. The within-emotion spread, on which our conclusions depend, is re-estimated in every subset.

Subsets drawn from the same data vary less than new samples would, so we scale their spread by √(0.8/0.2) = 2, the delete-d jackknife correction (Shao & Wu, 1989). We also confirmed empirically that 90% subsets lead to nearly the same standard errors (within 10%). We do not use bootstrap sampling, as sampling with replacement duplicates scenarios, and a duplicated scenario that appears in both halves of the within-emotion comparison leads to zero distance with itself, which artificially lowers the within-emotion spread. We report the difference from humans (model − humans) with 95% intervals:

ModelSeparation ratioBetween-emotion distanceWithin-emotion spreadImplicit–explicit ρ
QwQ 32B+0.60 [0.47, 0.76]+0.66 [0.44, 0.89]−0.80 [−1.12, −0.53]+0.18 [0.07, 0.30]
GPT o4-mini+0.79 [0.63, 0.96]+0.70 [0.47, 0.94]−1.03 [−1.35, −0.74]+0.34 [0.24, 0.46]
Phi-4+0.84 [0.67, 0.99]+0.67 [0.45, 0.89]−1.10 [−1.41, −0.81]+0.23 [0.12, 0.35]
Gemini 2.5 Flash+0.90 [0.72, 1.08]+0.73 [0.53, 0.96]−1.15 [−1.45, −0.84]+0.45 [0.35, 0.56]
DeepSeek R1+0.98 [0.80, 1.17]+0.76 [0.54, 0.99]−1.22 [−1.54, −0.92]+0.44 [0.33, 0.55]
Muse Glimmer 30B+1.11 [0.92, 1.32]+0.77 [0.54, 1.01]−1.34 [−1.68, −1.05]+0.37 [0.26, 0.48]
GPT-5.6 Luna+1.14 [0.96, 1.32]+0.73 [0.50, 0.95]−1.38 [−1.68, −1.10]+0.30 [0.19, 0.41]
Gemini 3.7 Flash+1.24 [1.04, 1.45]+0.78 [0.57, 1.02]−1.45 [−1.78, −1.17]+0.50 [0.39, 0.61]

No interval contains zero. In all 500 subsets, every model differs from humans in the same direction as in the full data. The structural differences between models and humans are therefore not explained by which participants or scenarios were sampled. We will add these intervals to the Appendix.

Reference: Shao, J., & Wu, C. F. J. (1989). A general theory for jackknife variance estimation. The Annals of Statistics, 17(3), 1176--1197.

(Q.1 (part b)) “A direct measure of within-emotion dispersion, such as the covariance trace…”

As suggested, we compute the trace of the covariance matrix of each emotion's standardised 17-dimensional appraisal representations, across the scenarios of that emotion. We average it over the 15 emotions, using the same scenarios and standardisation as the main analysis, and obtain 95% intervals for the (model - human) difference from the same 500 subsets described above:

ModelMean traceModel − humans [95% interval]Emotions lower than humans
Humans11.90——
Gemini 3.7 Flash5.07−6.83 [−8.57, −5.23]15 / 15
GPT-5.6 Luna5.26−6.63 [−8.34, −5.11]15 / 15
Muse Glimmer 30B5.41−6.49 [−8.27, −4.88]15 / 15
DeepSeek R15.61−6.29 [−8.00, −4.77]15 / 15
Phi-46.16−5.74 [−7.39, −4.08]15 / 15
Gemini 2.5 Flash6.16−5.74 [−7.43, −4.12]15 / 15
GPT o4-mini6.43−5.47 [−7.25, −3.91]15 / 15
QwQ 32B6.87−5.03 [−6.73, −3.50]15 / 15

The compression finding holds under this measure as well. Every model has a lower trace than humans on each of the 15 emotions, and no interval contains zero. The ordering of the models is the same as with the split-half Wasserstein spread, except that Phi-4 and Gemini 2.5 Flash, whose mean traces differ by 0.004, exchange places. We will add these results to Appendix F.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 3 - Continued from Above)

(Q.2 (part a)) “Clarification: Appendix H states that individual human ratings…”

Both of the analyses use the same pooling aggregation as described in Appendix H. We have corrected the caption and value in Table 11 in Appendix H. The different value for humans initially stemmed from an older computation on a subset of human participants, while the human study was still ongoing. We have thoroughly re-verified the computation, reproducing each result and re-running the whole pipeline on the final human data again to make sure all other values reproduce exactly. Thanks for flagging this!

(Q.2 (part b)) “What is the reliability of the human explicit dimension-importance judgments…”

We measure the reliability of the human explicit judgments and compare it with the models (6 sampled responses per emotion pair), also showing that the reliability does not affect our main conclusions:

HumansModels (range)
ICC(1,1), single judgement0.010.61-0.92
ICC(1,k), averaged judgement0.050.90-0.99
Rank agreement between two judges of the same pair (Spearman ρ)0.040.62-0.92
Split-half reliability of the averaged ranking (Spearman-Brown)0.150.88-0.98

Human participants therefore show little agreement on which appraisal dimensions distinguish two emotions, while each model gives highly consistent rankings across its sampled responses.

This limits how high the human explicit-implicit correspondence could be. The human implicit ranking has a split-half reliability of 0.72 (participants split into random halves). When two rankings have reliabilities of 0.15 and 0.72, the largest correlation they can show is √(0.15 × 0.72) = 0.33. The observed correspondence, 0.058, lies far below this bound, so the unreliability of the judgements does not by itself account for it. Dividing the observed value by this bound provides the correspondence expected if both rankings were perfectly reliable: 0.058 / 0.33 = 0.18, which remains below the uncorrected value of every model (0.24-0.55). The same correction for the models' explicit rankings raises their values by 1-6% (to 0.26-0.56), so the direction of the human-model difference is unchanged. We will add this interpretation to Appendix H.

(Q.3) “Does the training-stage finding persist when using expected ratings…”

We repeat the analysis with Olmo 3 training checkpoints with the expected rating, and report the separation ratios against the averaged human ratings (~1.29):

CheckpointMost probable ratingExpected rating
Pretrain1.932.44
Midtrain2.302.55
Base2.403.29
SFT2.283.07
DPO2.413.08
Instruct2.342.98

Under the expected rating, every checkpoint has a lower within-emotion spread than humans on all 15 emotions (Holm-corrected p < 0.001 at every stage) and a larger between-emotion distance (p ≤ 0.009). The separation ratio is highest at the base model, and post-training does not increase it. The training-stage finding therefore holds when the full distribution is taken into account. We will add these results and also note in Appendix O that, when using expected ratings, a large difference is also seen when going from the Midtrain to the Base checkpoint.

The narrowing of the distributions that the reviewer describes is what we report in Fig. 5(b), where the normalised entropy of the distribution over the rating scale decreases across training, most strongly with post-training, so the model becomes more decisive about its answer. As explained in Appendix O, we treat this decisiveness as a separate quantity and report the most probable rating, so that the rating itself does not depend on how decisive the model is. The additional expected-rating results show that post-training makes the model more decisive without making the emotion-level structure more categorical.

We hope these clarifications and the new results address the main points underlying the initial evaluation, and we would be very grateful if the reviewer would consider updating their score accordingly!

Reviewer_CiL5OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether LLMs organize emotion concepts the same way humans do. The authors build CoRE, a benchmark of 270 everyday scenarios covering 15 emotions rated on 17 appraisal dimensions such as how pleasant, fair, or effortful the situation was. Each emotion then becomes a set of points in appraisal space, for both humans and eight LLMs. For humans, scenarios of one emotion spread out and overlap with similar emotions, e.g., frustration and anger, while for LLMs they form tight clusters that sit far apart. LLMs also agree with each other more than different groups of humans do and this pattern also holds when the emotion label is removed and under different prompt framings, and it appears after midtraining rather than post-training.

Critical strengths

This is a strong paper. The question is interesting, the benchmark is carefully built, and the robustness analysis is thorough.

  • The paper is very well written and easy to follow.
  • The question is interesting and relevant. Looking at how emotion concepts are organized in latent space compared to only whether answers match humans can be useful in applications such as emotional support.
  • CoRe is thoughtfully constructed with multiple human validation stages and several people rating the same scenario.
  • The analysis is very thorough with several different measures and controls for human rating noise, prompt framing, and missing labels.
Critical weaknesses
  • Agreement between raters is low. The lower separability for humans could come from noisy ratings rather than real subjectivity.
Questions for authors

Each model answers each question only once at default temperature, and the Olmo results use the most probable rating. It would be interesting to see how model spread changes with multiple samples.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for their thoughtful engagement with our paper. We are glad that the reviewer found our benchmark carefully built and our analysis very thorough. We address the weaknesses and questions raised here:

(W.1) “Agreement between raters is low…”

The inter-rater reliability, as measured using the Intra-class Correlation Coefficient (ICC(1,k), k = 3.7), indeed remains modest (~0.44). However, we understand and measure its impact in two ways:

  • Modest inter-rater reliability represents systematic individual variation instead of measurement noise: In Appendices C.2, C.3, C.4, we verify three things: that individual differences in how scenarios are appraised replicate consistently, that appraisal ratings obtained align with theoretically predicted associations from human psychology, and that the mean appraisal profiles for a given emotion remain stable across all human participants.
  • The residual measurement noise for humans does not explain the human-model difference: For each of our main results, in the corresponding appendices, we include results where the same measurement error in humans is introduced in models. The higher categorical nature of LLMs holds under this control, for every set of results. In Appendix F, we also recompute the human appraisal distributions by averaging more raters (Table 7), showing that even when considering a perfectly reliable panel of human raters, LLMs remain more categorical than humans (separation ratio remains higher for models).

(Q.1) “Each model answers each question only once….how model spread changes with multiple samples.”

We draw 3 additional independent response sets from two models at opposite ends of our model range (Gemini 3.7 Flash and Phi-4), with the same prompts and default generation settings as in the main experiments. We compare the variation between a model's responses to the same question with the variation between human raters of the same question:

Gemini 3.7 Flash (4 samples)Phi-4 (4 samples)Humans
SD of responses to the same question0.280.632.53
Identical responses (pairs of responses to the same question)76%52%14%
Separation ratio, one response per question2.53 to 2.592.11 to 2.131.09 (one rater)
Separation ratio, averaged responses2.68 (4 responses)2.38 (4 responses)1.29 (3.7 raters on average)

A model's responses to the same question vary far less than different humans' ratings do, so a single response per question measures the model's representation with high reliability (mean ICC(1,1) over the 17 dimensions, for Gemini = 0.97, for Phi = 0.88). Models also remain more categorical than humans in each independent run, and when the independent ratings are averaged.

For the Olmo checkpoints, the result with the expected rating, which uses the full probability distribution instead of the most probable rating, is reported in our response to Reviewer UZFo. Every checkpoint there too remains more categorical than humans.

We hope the clarifications and additional results address the main concerns raised. We are happy to answer any further questions during the discussion period.

AuthorsOpenReview ↗

Request to Engage

Dear Reviewer,

As the discussion period is coming to an end soon, we were wondering if our rebuttal addresses your current set of concerns. We are happy to provide further clarifications or answer any other questions if there are any remaining concerns.

Thank you for the thorough response and the additional multi-sample experiments. I will maintain my clear accept score.

AuthorsOpenReview ↗

Thank you!

Thank you for your response! We are very glad that our response addressed the main concerns raised!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summarizing the Rebuttal

Dear AC and reviewers,

We firstly thank the reviewers for their meaningful and continued engagement with our paper, and for their thoughtful comments that have helped make the paper stronger. We summarize here how we clarified the main concerns raised by each reviewer during the rebuttal process.

Reviewer CiL5:

  • Lower agreement between raters: we clarified that although the agreement indeed remains modest in absolute terms, we verified that this stems mainly from systematic individual variability, and the residual measurement noise did not affect our main conclusions.
  • How model spread changes with repeated sampling: we add new results with two models, showing that a single model response per question can be used to reliably estimate the appraisal representation for models.

The reviewer started with an initial rating of 4 and maintained their rating.

Reviewer UZFo:

  • Clarification of novelty against specific past research: we provided extensive clarifications of how our work is novel compared to the cited papers, in that our goal is to measure the structural alignment of emotion concepts; our methods are distinct in holding the emotion fixed and eliciting first-person appraisals from models and humans on the same scenarios, rather than predicting emotions or appraisals; and our findings, of more categorical behavioral representations, are, to the best of our knowledge, novel.
  • Uncertainty intervals for main structural differences, across sampling different participants and scenarios: we provided additional analyses showing that the main structural differences between models and humans hold significantly (intervals exclude 0) even across random subsamples of participants and scenarios. We also showed that our main conclusions hold when using a different measure (covariance trace) to measure the within-emotion dispersion.
  • Reliability of human explicit judgements: we showed through additional analyses that human participants show little agreement on which appraisal dimensions distinguish two emotions. However, we also showed that the unreliability of the judgements does not by itself account for the low implicit-explicit correspondence, and that the direction of human-model difference holds even when correcting for the unreliability of the human explicit judgements.
  • Training-stage findings on Olmo checkpoints using the full rating distribution: we confirm through additional analysis with the expected ratings that every checkpoint remains more categorical than humans. We also show that the separation ratio is highest at the base model, and post-training does not increase it, broadly aligned with our main results.

The reviewer started with an initial rating of 1, but affirmed that they will raise their final rating to 3, given the additional analyses and clarifications.

Reviewer hJgS:

  • Comparing models with individual human responses instead of averaging across participants: we showed with additional analyses that a stronger version of our main conclusion holds, with human representations becoming even less categorical.
  • Using scale-free methods for self-reports: we showed through additional analysis that our main conclusions about the human-model difference hold even when appraisal ratings (on a Likert-like scale) are replaced with ordinal information only (ranks).
  • Effect of explicit CoT reasoning: we added new experiments with two models where explicit CoT reasoning is added to our vanilla setup. We find that explicit CoT reasoning does not increase discretization and leaves both models far more categorical than humans.

The reviewer started with a rating of 3 and raised their final rating to 4 given the new empirical evidence provided in the rebuttal.

Reviewer Kqua:

  • Alignment in external behavior vs. alignment in internal representations: we agreed with the reviewer that we study the former. We clarified where in the paper we state explicitly that we study alignment in external behavior, and we will also modify a few additional phrases to remove any confusion.
  • Comparison of human-authored scenarios and GPT-authored scenarios: we showed that the main conclusions hold in both subsets. In response to the reviewer’s follow-up on the human metrics moving in opposite directions across the subsets, we showed that this mainly reflects the smaller size of the human-authored subset. At equal size, the subsets do not differ in separation for humans, with at most a modest tendency for humans to classify the human-written scenarios more accurately.

The reviewer started with a rating of 3 and maintained their rating after the rebuttal.

We thank all reviewers and the AC for the constructive comments, which have led to more meaningful analyses and experiments! We will incorporate all changes in the paper as suggested during this period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.