- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies how emotion probes depend on emotion taxonomy, stimulus domain, and read-out position. On two models, it compares three taxonomies using AI-centric stimuli at the prompt's last token through selectivity, probe-weight cosine similarity, and RSA. Across seven models, it compares three domains and two read-out positions with Ekman labels using selectivity and probe-weight cosine similarity. The results suggest similar structures across taxonomies, but substantial differences across domains and positions, with greater cross-domain alignment after generation.
Critical strengths
- The paper tries to address a relevant methodological question about whether emotion probes developed using human scenarios remain informative in situations concerning the model itself.
Critical weaknesses
- The claim that AI-centric emotion representations arise only during generation exceeds the evidence. Figure 6 already shows above-chance decoding before generation, despite the conclusion stating that a probe-able representation "only appears during the generation." Probing intermediate response positions and responses re-encoded without their prompts would help distinguish changes during generation from information in the response text used for labeling.
- The claim that prompt-position representations do not generalize across domains lacks direct transfer evidence. Low cosine similarity between separately trained probes does not establish unrelated representations, and cross-taxonomy comparisons on the same activation and response sets are not a substitute for a probe-stability baseline. Cross-domain prediction tests and within-domain retraining controls are needed.
Questions for authors
- The generation pipeline does not sufficiently isolate the effect of whether a scenario concerns an AI or a human. Could the authors clarify how the domain comparison accounts for the differences in speaker role and register between Listings 5 and 7, as well as the response styles they may elicit?
- The suitability of GPT-5-mini as the sole annotator has not been sufficiently validated. Could the authors provide evidence of primary-label reliability across all three domains and taxonomies, given that Appendix A evaluates label-set agreement on only 132 AI-centric responses under Ekman and GoEmotions?
- Differences in class distribution and training sample availability limit the interpretation of the cross-domain selectivity comparisons. Could the authors report primary-label class counts across all seven models and clarify whether the cross-domain selectivity differences persist after accounting for class support?
- The recommendation that emotion taxonomy can be chosen on practical grounds is broader than the evidence. Could the authors clarify what supports extending this recommendation to other domains and read-out positions, given that the taxonomy analysis covers only two models with AI-centric stimuli at the prompt position?
5 comments
Response to Reviewer qE5k (2/3)
Could the authors clarify how the domain comparison accounts for the differences in speaker role and register between Listings 5 and 7, as well as the response styles they may elicit?
The question allows us to clarify the design. The three domains cross two factors: content (whether the situation is that of an AI or of a human) and register (a message addressed to the assistant or a vignette narrated in the third person).
| Addressed to the assistant | Narrated in the third person | |
|---|---|---|
| AI situation | AI-centric | — |
| Human situation | Human conv. | Human 3rd |
Each pair of domains approximately isolates one of the factors. Human 3rd and human conv. share the content and differ only in register, while AI-centric and human conv. share the register and differ in content. We acknowledge as the main limitation of the design the absence of the fourth cell, the one of AI situations narrated in the third person, which would fully separate the two factors.
Regarding response styles, the data show that they differ greatly between domains. Faced with AI-centric stimuli, the models respond with disclaimers (e.g. "as an AI, I don't have a physical core…") and/or a neutral or helpful tone, consistent with the observation by Sofroniew et al. that post-training moves the assistant toward "a more measured, contemplative stance" [1]. Faced with third-person vignettes, they continue the story, in a style close to that of a base model. Faced with messages from a user, they respond with empathy, also aligned with the assistant persona that models adopt [2]. The distribution of labels reflects these styles: neutral is the majority class in AI-centric in 6 of 7 models, while in human 3rd the majority class is sadness in all 7 (we detail the counts below).
Our reading in this regard is that when the model identifies a highly emotionally charged situation directed at itself, it responds almost by default in a neutral or positive way, given that negative responses about itself were probably penalized during training. That is why we do not see the differences in register and response style as confounders to be eliminated, but as the phenomenon we are interested in characterizing. The question of the paper is precisely whether probes built on human situations remain informative when the situation involves the model itself, and the answer depends on the fact that the model reacts differently to those situations. A design that matched response styles across domains would eliminate the difference we seek to measure.
The suitability of GPT-5-mini as the sole annotator has not been sufficiently validated. Could the authors provide evidence of primary-label reliability across all three domains and taxonomies
We thank you for the suggestion, which led us to run an agreement test with a second annotator. Using Claude Haiku 5.5, we first measured agreement on the 132 expert-annotated responses. There, the second judge reaches an agreement comparable to that of the judges in Table 1 of the submitted version (Table 2 in the revised one) (α MASI = 0.36 in Ekman, versus 0.30–0.37 for the panel and 0.35 for GPT-5-mini), even though the rubric was not tuned to calibrate this model's annotations.
Then we measured the reliability of the primary label across the three domains and the three taxonomies, where the second judge re-annotated ~27,000 responses from the 7 LLMs (~9,000 per domain) with the same rubric. Below, the agreement with GPT-5-mini on the primary label (Cohen's κ):
| Taxonomy | AI-centric | Human 3rd | Human conv. |
|---|---|---|---|
| Ekman | 0.69 | 0.70 | 0.74 |
| Plutchik | 0.46 | 0.64 | 0.59 |
| GoEmotions | 0.51 | 0.50 | 0.49 |
In Ekman, the taxonomy used for the analyses on read-out position and domains with the 7 models, agreement is substantial and practically the same across the three domains, so annotation noise cannot explain the differences between them.
As context for the problem of emotion annotation in general, it is reported in previous literature that the task has moderate agreement among human annotators as well. Inter-annotator κ in the range of 0.04 to 0.57 depending on the emotion and the pair of annotators with Plutchik's categories [3], and a median of 0.25 across the emotions of GoEmotions [4].
We consider that these results help strengthen the annotation process, so we included them in Appendix A of the paper.
Response to Reviewer qE5k (3/3)
Could the authors report primary-label class counts across all seven models and clarify whether the cross-domain selectivity differences persist after accounting for class support?
We added the primary-label counts by class, model and domain for the three taxonomies to the revised version of the manuscript (Appendix D). In Ekman, the distributions follow a clear pattern that repeats within each model. In AI-centric, neutral is the majority class in 6 of 7 models (48–70%), followed by enjoyment (24–43%). In human 3rd, the majority class is sadness in all 7 models (38–54%), and in human conv. sadness (35–45%) and enjoyment (28–42%) account for most responses.
On the other hand, the probe pipeline already incorporates several corrections for class imbalance. In training, the class weights of the logistic regression are inversely proportional to the frequency of each class, and the train/test split is also stratified. As for the results, the reported metric is macro-F1, so correct predictions on the majority class do not inflate the result. Moreover, the selectivity metric is computed against a shuffled-label control with exactly the same class distribution, which discounts the difference in chance level between domains.
The recommendation that emotion taxonomy can be chosen on practical grounds is broader than the evidence. Could the authors clarify what supports extending this recommendation to other domains and read-out positions
We extended the taxonomy analysis to the three taxonomies, in the three domains and the 2 positions, for Llama-2-7b and Qwen2.5-14B (included in the Appendix D). The qualitative patterns, and therefore our conclusions about domain and position, hold with any of them. At the prompt, AI-centric falls below the two human domains in all 6 cases (2 models × 3 taxonomies). From the prompt to the generation read-out, AI-centric increases in all 6 cases, while the human domains decrease in 11 of 12. The only exception, Qwen2.5 with Plutchik in human conv., increases by 0.009, practically no change.
Also, if we compare the selectivity profiles across layers for each pair of taxonomies, at the prompt read-out the profiles of the three taxonomies are almost perfectly correlated in the three domains (r ≥ 0.93). The same occurs at the generation read-out, where the correlation remains high in 5 of the 6 cases (r ≥ 0.84), and the exception is Qwen2.5 in AI-centric, with a moderate correlation (r = 0.70–0.80). That is, the three taxonomies recover the same structure across the depth of the model in all domains and positions, although they slightly differ in the precise level of decodability.
We acknowledge that this analysis covers two models, but we consider it a representative sample of the 7 models, given that the results suggest that Llama-2-7b and Qwen2.5-14B lie at the extremes of emotional expressiveness among the chosen LLMs.
We thank you again for your comments, which substantially improved the work. If these clarifications and the new analyses address your concerns, we kindly ask you to consider raising your score.
References:
[1] Sofroniew, N., Kauvar, I., Saunders, W., Chen, R., Henighan, T., Hydrie, S., ... & Lindsey, J. (2026). Emotion concepts and their function in a large language model. arXiv preprint arXiv:2604.07729.
[2] Lu, C., Gallagher, J., Michala, J., Fish, K., & Lindsey, J. The assistant axis: Situating and stabilizing the default persona of language models, 2026. URL https://arxiv. org/abs/2601.10387.
[3] Schuff, H., Barnes, J., Mohme, J., Padó, S., & Klinger, R. (2017, September). Annotation, modelling and analysis of fine-grained emotions on a stance and sentiment detection corpus. In Proceedings of the 8th workshop on computational approaches to subjectivity, sentiment and social media analysis (pp. 13-23).
[4] Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020, July). GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 4040-4054).
Response to Reviewer qE5k (1/3)
Dear Reviewer qE5k,
We thank you for a detailed review, with concrete suggestions that guided a good part of the new analyses. We are glad that you recognize that the work addresses "a relevant methodological question about whether emotion probes developed using human scenarios remain informative in situations concerning the model itself". We respond to each point below.
It is correct that the sentence in the conclusión, by stating that emotion representation only appears during generation, contradicted Figure 6. We nuanced the claims. The revised version claims that the tone of AI-centric responses is less predictable before generation than that of responses to human stimuli, and also that a significant portion of the emotional tone is determined during generation, although it is already decodable above chance at the prompt.
Furthermore, we ran extra analysis using a text only baseline (TF-IDF with logistic regression), which supports our reading. The text of AI-centric prompts predicts the tone of the response worse than that of human prompts in all 7 LLMs, while moving from the prompt text to the response text increases selectivity by a similar amount in the three domains (mean across the 7 models: 0.16 to 0.37 in AI-centric, 0.27 to 0.50 in human 3rd and 0.26 to 0.51 in human conv.).
The main claim holds, which is the interaction between domain and position. We added a formal test to support this claim. The change from the prompt to the generation position is more favorable for AI-centric than for both human domains (exact sign test, p = 0.016), although the AI-centric increase on its own is not consistent (5 of 7 models).
We implemented the cross-probing analysis between domains. We applied each Ekman probe trained at the prompt position to the test-split activations of the other two domains, using the five plateau layers of each of the 7 LLMs. Below, the mean selectivity across the 7 models:
Off the diagonal, the transfer is above chance in all cases, so we interpret that the probes of different domains are not independent, and we nuance this statement in the revised version of the paper. However, transfering loses a large part of the decodability. A probe trained on human situations and applied to the AI-centric domain retains between 42% and 51% of the selectivity reached by a probe trained on AI-centric, while between the two human domains between 60% and 67% is retained. On the other hand, the AI-centric probe transfers relatively well to human conv. (0.24), but worse to human 3rd (0.16).
Overall, we nuance the support given by the cosine, since results show decodability above chance in the cross-probing analysis. However, we still claim that an emotion probe does not fully generalise to other domains.