- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper aims to characterize emotion concepts through behavioral representations for LLMs that can be compared with human better directly. The authors build the CoRE benchmark on cognitive appraisal theory. Based on the representation and the benchmark, the paper finds that LLMs and humans share a broadly similar overall structure, but LLMs organize emotion concepts in a way that is more categorical (different emotions lie further apart and instances of the same emotion cluster more tightly), more determinate (emotions can be distinguished more accurately from the ratings alone), and more convergent (different models agree with one another more than different groups of humans do). These properties persist across changes in prompt context and are already present before post-training. The authors conclude that LLMs lack the context-dependent variability that characterizes human emotion.
Critical strengths
-
The paper sets out to develop a characterization of emotion concepts that allows direct comparison between humans and LLMs, which is a valuable goal. This characterization is grounded in classic cognitive appraisal theory.
-
Building on this representation, the paper conducts a systematic analysis from multiple perspectives. Its core findings — that in LLMs, different emotions lie further apart, instances of the same emotion cluster more tightly, and emotions are easier to distinguish — are supported by multiple metrics, including Wasserstein distance, logistic regression, and SynthTree, all of which converge on the paper’s central result regarding structural alignment. The experiments are fairly thorough.
Critical weaknesses
W1. Mismatch between the level of the conclusions and the methodology.
The abstract motivates the study by asking “whether LLMs internalize emotion concepts in a way similar to humans,” and the title and conclusions state that LLMs represent emotion concepts more categorically. However, This paper does not attempt to locate the directions or subspaces in which models internally encode emotion information. Instead, it compares the explicit ratings that models and humans give on a "predefined questionnaire". In my view, alignment in external behavior and that in internal representation are questions at two different levels. The current evidence therefore supports only the claim that response patterns on the appraisal questionnaire differ; it is not sufficient to establish that the internalized emotion concepts differ. I suggest that the authors narrow their claim.
W2. Potential circularity in dataset construction.
Source bias: most of the scenarios in CoRE were generated by GPT-4o expansion (Stage 2 of the four-stage curation pipeline; Section 2, Fig. 6). The only human-written scenarios are the seed scenarios from Stage 1 and the alternative scenarios provided by annotators in Stage 3. Meanwhile, the evaluated models include GPT-family models. I am concerned that this construction may introduce a systematic bias into the evaluation results, since LLMs are effectively being evaluated on LLM-generated material.
Questions for authors
Q1. To what extent do the observed differences between humans and LLMs stem from biases in the dataset itself? I would like the authors to report the proportions of human-written and GPT-4o-generated scenarios among the final 270 scenarios. If possible, I would also like them to repeat the main analyses (separation ratio and classification accuracy) separately on the human-written and GPT-4o-generated subsets, to check whether the conclusions remain consistent with those reported in the main text.
4 comments
Request to Engage
Dear Reviewer,
As the discussion period is coming to an end soon, we were wondering if our rebuttal addresses your current set of concerns. If it does, we would be very grateful if you could consider updating your score accordingly. Otherwise, we are happy to provide further clarifications or answer any other questions if there are any remaining concerns.
Thank you for the detailed response and the additional analyses. My two main concerns have therefore been largely addressed, and I will maintain my score of 3 (weak accept).
One observation from the new results may be worth a brief comment. For human raters, the two metrics move in opposite directions across the subsets: on the human-written scenarios, the separation ratio is lower (1.21 vs. 1.32), yet the classification accuracy is higher (0.40 vs. 0.27). One would usually expect less separable categories to be harder to classify. It would be helpful if the authors could explain this discrepancy, for example whether it reflects the small size of the human-written subset (94 scenarios, about six per emotion), and whether it still holds with a larger set of human-written scenarios.
Either way, I find this an interesting result.
Thank you!
Thank you for your response! We are glad the rebuttal largely addressed both of your concerns!
We also thank you for the observation about the diverging metrics. As suggested, the size of the human-written subset plays a central role, and we examined both metrics with the two subsets compared at equal size.
For the separation ratio, when both subsets are drawn down to the same number of scenarios per emotion (200 random draws), the human ratio is 1.21 on the human-written and 1.24 on the generated scenarios (difference −0.03, 95% range −0.11 to 0.04). The lower ratio on the human-written subset therefore reflects its smaller size.
For classification accuracy, we repeated the cross-validation with 100 different random divisions into folds and averaged, for each scenario, how often it is classified correctly. On average then human accuracy is 0.41 on the human-written and 0.31 on the generated scenarios. This difference is at the margin of statistical significance (permutation test, reassigning scenarios between the two subsets within each emotion: p = 0.06), and it does not appear when the classifier is trained separately on each subset with the same number of scenarios per emotion (human accuracy 0.37 and 0.41; difference −0.04, 95% range −0.15 to 0.06). The difference between models and humans, in contrast, holds in every division: every model exceeds humans by at least 0.17 on the human-written and 0.29 on the generated scenarios.
At equal size, the two subsets therefore do not differ in separation for humans, and humans show at most a modest tendency to classify the human-written scenarios more accurately. The difference between models and humans, which is central to our main conclusions, remains large in both subsets. We will include this additional result and interpretation in the appendix. Thank you!
Rebuttal by Authors
We thank the reviewer for their thoughtful engagement with our paper. We are also glad that the reviewer found our core findings convergent and our experiments thorough. We respond to each of the weaknesses and questions raised below:
We agree that alignment in external behavior and alignment in internal representations are questions at different levels, and our study addresses the first. The paper states this at several points:
We will additionally revise two specific phrases in the abstract and introduction, which ask whether LLMs "internalize" emotion concepts, and can be interpreted as a claim about internal representations. We will state further explicitly in the introduction and conclusion that our findings concern behavioral representations elicited through appraisal judgments, measured at a different level from studies of internal model representations.
Within the final scenarios, 94 (~35%) are directly human-authored, across the first two stages of data curation. Additionally, 72% of the scenarios in the final set are rewritten or edited lightly in the final stage of verification by the research team (Stage 4).
We repeat the separation-ratio and classification analyses separately on the purely human-authored scenarios and the remaining scenarios. Classification uses the logistic classifier from the main text, with its accuracy computed separately on each subset.
Our main conclusions remain the same in both subsets. Every model has a smaller within-emotion spread than humans (Holm-corrected p < 0.005 in both subsets), a larger between-emotion distance (p < 0.008 in both subsets), and a higher per-emotion F1 (p < 0.03). Retraining the classifier within each subset leads to the same result (humans 0.38 and 0.35; models 0.63-0.81 in both subsets). Humans classify the human-written scenarios slightly better than the generated ones, while models do not show a consistent difference, so the human-model gap narrows slightly on the human-written scenarios but remains large. The observed differences are therefore not explained by the origin of the scenarios. We will add these results to Appendix B.2.
We hope these clarifications and the new results address the main points underlying the initial evaluation, and we would be very grateful if the reviewer would consider updating their score accordingly!