- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates how emotional information is represented, attended to, and transformed during response generation in large language models. Using Llama-3.2-3B-Instruct and the EmpatheticDialogues dataset, the authors conduct three analyses: (1) layer-wise linear probing with L1 and L2 regularization to examine emotion representations, (2) attention analysis to investigate whether emotion-related tokens receive preferential attention during generation, and (3) emotion trajectory analysis to measure changes in emotion alignment between user inputs and generated responses. The results suggest that emotion information is most accessible in middle layers, emotion-related tokens receive greater attention during generation, and generated responses generally exhibit reduced alignment with the original target emotion, particularly for negative emotions. The paper aims to provide a framework for understanding the relationship between emotion representation and empathetic response generation in LLMs.
Critical strengths
- This paper proposes a relatively unified interpretability analysis framework for empathetic dialogue, extending emotion analysis from classification to generation while maintaining a good level of consistency.
Critical weaknesses
-
The causal relationships underlying some experimental settings require further clarification, and the manuscript exhibits noticeable signs of AI-assisted writing.
-
The paper overlooks several relevant studies that may affect the assessment of its novelty.
Questions for authors
-
Missing citations to relevant prior work. The paper omits several relevant studies. For example, the contrastive directions introduced in Section 2.3.3 are conceptually similar to the Concept Activation Vectors discussed in “FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks” (ICML 2025). Similarly, the two regularization approaches employed in Section 2.3.1 are closely related to the methodology explored in “Discovering Salient Neurons in Deep NLP Models” (Durrani, Dalvi, & Sajjad, Journal of Machine Learning Research, 2023). These related works should be properly discussed and cited to better contextualize the paper's methodological contributions and novelty.
-
Limited generalizability of the experimental findings. The experiments are conducted exclusively on Llama-3.2-3B, without further evaluations across different model architectures or scales. Additional experiments are needed to assess the generalizability and robustness of the reported findings.
-
Insufficient justification for emotion category selection. The authors select 10 emotion categories from the original set of 32 but do not adequately explain the rationale behind this selection. It remains unclear why these particular 10 categories were chosen instead of considering the full set of 32 emotions.
-
Potential sensitivity to emotion representation strategies. The choice of emotion representation may require further investigation. Di Palma et al. (2025) explored multiple representation and pooling strategies for emotion-related probing tasks. It is unclear whether alternative representation strategies would substantially affect the experimental results and conclusions of this paper. An ablation study comparing different pooling strategies would help establish the robustness of the findings.
-
Insufficient justification for transferring probing results to response-generation analysis. In the proposed framework, the representation strategy and model layer identified as optimal for emotion classification are directly adopted as the basis for analyzing response generation. However, a layer that contains highly decodable emotion-related information does not necessarily contribute to the generation of sufficiently empathetic responses. The relationship between emotion decodability and empathetic response generation requires further investigation, ideally supported by causal evidence.
-
Unclear relationship between representational drift and actual emotional change. In the emotion drift experiments presented in Section 2.3.3, the authors primarily observe changes in emotion-related representations. However, it remains unclear whether such representational shifts correspond to meaningful changes in the emotional content of the generated responses. Additional validation is needed to establish the relationship between representational drift and actual emotional change, rather than assuming that changes in hidden-state alignment necessarily reflect changes in emotional content.
-
Noticeable signs of possible AI-generated content in Figure 1. The upper portion of Figure 1 exhibits visual artifacts that appear consistent with AI-generated imagery, potentially produced by an image-generation model such as GPT Image. In particular, noticeable irregular white regions can be observed around text labels such as “Fear” and “Anger.” The authors should clarify how this figure was created and ensure that the visual presentation meets academic standards.
-
Unresolved placeholder in Appendix A.8. Appendix A.8 contains an obvious placeholder: “insert the exact observed pattern and supporting values.” This suggests that the manuscript has not been thoroughly proofread and that the corresponding analysis or description remains incomplete.
Thank you for your valuable comments on our manuscript. We appreciate your suggestions and will incorporate them, along with additional proofreading, in a revised version.