acceptodds
Under review as a conference paper at ICLR 2027

PSYCHE: Modeling Conversational Emotion as a Continuous Trajectory via Causal Source Decomposition

Abstract

Multimodal Emotion Recognition in Conversation (MERC) faces a fundamental challenge: textual, acoustic, and visual expressions are sparse, regulation-filtered observations of internal emotional states that evolve continuously. Genuine calm and suppressed disappointment can exhibit nearly identical surface signals yet arise from opposite trajectories. Existing methods harness rich conversational context and multimodal cues, yet model emotion as a discrete per-utterance label rather than an evolving state. Shifts are consequently detected by auxiliary binary heads, while causes are attributed post hoc by separate modules, so that how emotion evolves and why it changes are never decoded by a single process. We propose PSYCHE, which models each speaker's emotion as a latent trajectory governed by a Neural Jump ODE: Causal Source Decomposition (CSD) structures the driving force for continuous inter-utterance evolution, while speech acts trigger discrete updates. CSD decomposes this driving force into a self-cause driver with state-dependent decay and an other-cause driver attending to the interlocutor's triggering utterances, fused by a per-dimension gate. The self-/other-cause balance and trigger identification are computed by the forward pass itself, requiring no post-hoc explainer or inference-time language model. On IEMOCAP and MELD, PSYCHE achieves the best weighted F1 among the baselines we re-evaluate under identical multimodal features, with offline commonsense cues as an added input. Dynamics analyses show that the model lowers the self-cause ratio when emotion turns and reduces confusion precisely where surface signals mask internal states, indicating that how and why are not two answers but one mechanism.

Primary area:
interpretability and explainable AI
Keywords:
Emotion Recognition in Conversation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cNzXOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PSYCHE, a multimodal conversational emotion recognizer that models each speaker's emotion as a continuous latent trajectory. Its Causal Source Decomposition separates emotional inertia from influence by the interlocutor, and a Neural Jump ODE uses these drivers to update the state between utterances while speech acts create discrete jumps. The model also outputs a self/other balance and trigger attention. On IEMOCAP and MELD, it reports weighted F1 scores of 72.51 and 67.55, with additional ablations and attribution analyses.

Critical strengths
  1. The core idea is easy to grasp: emotion can carry over between turns and also be affected by the other speaker.
  2. CSD makes those two sources visible in the model instead of hiding them in one blended context vector.
  3. The causal mask and pre-speech state design are thoughtful choices for avoiding future-turn leakage.
  4. The paper includes useful ablations and dynamics analyses, not just benchmark scores.
Critical weaknesses

1.The main comparison gives PSYCHE extra offline commonsense features. Without both cues, its IEMOCAP W-F1 is 70.74, below GraphSmile's 71.32, so the headline gain is hard to attribute to the new architecture alone. 2.The paper calls the outputs causal attributions, but the evidence is mostly correlations with annotations. Attention weights and a learned gate do not by themselves show that a particular utterance caused the emotion change. 3.The gate is trained with an LLM-derived prior and losses that encourage lower self-ratios at shifts. That makes the later shift analyses less independent as evidence for the gate's meaning. 4.It is still unclear what the ODE adds beyond CSD and its losses. Removing the ODE costs 2.30 W-F1 points, while removing CSD costs 4.97; a matched discrete-time version would help isolate the contribution. 5.The main table uses the best validation checkpoint, while the reported three-seed mean is lower. Paired significance tests or consistent multi-seed results for the strongest baselines would make the small margins easier to judge. 6.The RECCON trigger results need more context in the main paper, especially the number of evaluated cases, subset construction, and variation across seeds. 7.The offline LLM features and attribution prior could affect results in ways that are hard to reproduce or audit. The paper should make the exact prompts, visible context, filtering, and human checks easy to verify.

Questions for authors
  1. Can you provide an apples-to-apples comparison in which the strongest baselines receive the same entity- and scene-level commonsense features, as well as a clear comparison using no commonsense features? Please state exactly what dialogue context was visible when the offline LLM generated each feature.
  2. Can you add a discrete-time baseline that preserves the self/other split, CSD-style drivers, and training losses but removes the Neural Jump ODE? This would isolate whether the continuous-time formulation contributes beyond the source decomposition and its auxiliary objectives.
  3. For the RECCON trigger evaluation, how many transitions and candidates are used, how is the evaluated subset selected, and what are the mean and standard deviation across seeds? Please also clarify whether any trigger labels or RECCON-derived information influence training, including the offline attribution prior.
4 comments
AuthorsOpenReview ↗

Response to Reviewer cNzX [1/2]

We thank the reviewer for the careful reading and the positive assessment of our core idea. We are glad it came across as easy to grasp, because that is exactly how we arrived at it: we asked how a person would explain an emotion change, and the continuous ODE was simply the natural way to carry that explanation, keeping the model small. We have added the requested controlled comparisons and disclosures, as detailed below.


[W1/W7/Q1] Fairness of the comparison with the LLM commonsense cues.{\color{#f26921}\text{[W1/W7/Q1] Fairness of the comparison with the LLM commonsense cues.}}

1. Fair comparison.

To examine whether the headline gain comes from the architecture itself or from the extra commonsense cues, we add two controlled comparisons.

(i). Granting the strongest baselines the same cues. We retrain GraphSmile and ECERC with the same commonsense cues, as reported in STable 5, Appendix, Sec. 3.7:

Methodsw/o com.w/ com.Δ
GraphSmile71.3272.28+0.96
ECERC70.2470.45+0.21
PSYCHE70.7472.51+1.77

(STable 5: IEMOCAP W-F1 %. com. = commonsense cues.) The commonsense cues help GraphSmile by +0.96+0.96 and leave ECERC nearly unchanged, at +0.21+0.21, while PSYCHE still leads at 72.5172.51.

(ii). The CSD gain is independent of the cues. A 2$\times$2 cross-ablation that varies the commonsense cues and CSD isolates the CSD contribution, as reported in Table 6, Sec. 4.3, around L.544:

Settingsw/ CSDw/o CSDCSD gain
w/ commonsense72.5167.54+4.97
w/o commonsense70.7466.33+4.41

(Table 6: IEMOCAP W-F1.) The CSD gain is +4.97+4.97 with the cues and +4.41+4.41 without them, i.e., CSD contributes to accuracy independently of the cues. This does not make the cues redundant: they serve a complementary role, the situational grounding that separates, e.g., composure at a funeral from composure that masks disappointment. This is why knowledge enhancement is a common practice in MERC, such as MKE-IGN [1]^{[1]} and KAMIN$^{[2]}$.

Together, the two experiments show that the headline gain comes from the architecture, not the added cues. To be transparent, we also state in the manuscript that without any cue, PSYCHE reaches 70.7470.74, slightly below GraphSmile's 71.3271.32. This gap does not change the core contribution. The cues provide what the conversation is about, whereas CSD provides the source-level interpretability the cues cannot substitute. Its self-cause ratio and trigger attention, validated against RECCON in our reply to Q3, decompose how and why an emotion changes and are read directly off the forward pass without any language model. This interpretability, not a raw classification margin, is the core contribution.

2. Audibility of the LLM-generated cues.

Regarding the LLM context used to generate the cues, the entity-level cue is generated from the full dialogue transcript, and the scene-level cue from only the dialogue-onset background, namely the speaker identities, the setting, and the power dynamics, with no future utterance outcome visible. The exact prompts, JSON schemas, and extraction scripts are released in the Appendix 2.3.


[W2] The attribution is correlational, not interventionally causal.{\color{#f26921}\text{[W2] The attribution is correlational, not interventionally causal.}}

We appreciate this comment on the evidence for the attribution. We agree that the attention weights and the gate are correlational rather than causal: they localize the utterance most associated with an emotion change, and we validate this localization against human annotations, but we do not claim that they prove, in the intervention sense, that this utterance caused the change. In the paper, "causal" modifies the source decomposition, and the attribution is source-level; we did not intend "causal attribution" as a claim of intervention-level causation. Sec. 3.2 states the scope explicitly as time-admissibility of causal effect rather than causal discovery. Φtself\Phi^{\rm self}_t and Ψtother\Psi^{\rm other}_t are drivers, the two source-conditioned inputs to the Neural Jump ODE, and the only causal assumption we rely on is time-admissibility, that a cause precedes its effect in time and is attributable to a source. We validate the localization against independent human annotations of the emotion cause, RECCON, the evidence appropriate for a recognition model whose aim is interpretability.


References

[1] Geng Tu, Jun Wang, Zhenyu Li, Shiwei Chen, Bin Liang, Xi Zeng, Min Yang, and Ruifeng Xu. Multiple Knowledge-Enhanced interactive graph network for multimodal conversational emotion recognition. In ACL Anthology Proceedings, pp. 3861–3874, 2024.

[2] Minjie Ren, Xiangdong Huang, Jing Liu, Zan Gao, Yuting Su, and An-An Liu. Ka-min: Knowledge-aware multimodal interaction network for emotion recognition in conversation. IEEE Transactions on Circuits and Systems for Video Technology, 2025.

AuthorsOpenReview ↗

Response to Reviewer cNzX [2/2]

[W3] The gate ordering is learned, not imposed by the losses.{\color{#f26921}\text{[W3] The gate ordering is learned, not imposed by the losses.}}

We appreciate this comment. We agree that the gate is trained with an LLM-derived prior and losses that encourage lower self-ratios at shifts. To check whether this training makes the shift analysis less independent, we remove these losses and test whether the shift-vs-stay ordering survives. It survives: a variant trained without both Lattr\mathcal{L}_{\rm attr} and Lgate\mathcal{L}_{\rm gate} still gives gˉt\bar{g}_t of 0.3860.386 at shifts versus 0.5400.540 at stays, a gap of 0.1540.154 with p=6.53×10−24p=6.53\times10^{-24}, nearly matching the full model's gap of 0.1670.167 (Figure 6, see the revised manuscript, around L.504). So the independence is not reduced: the shift analysis remains independent of these losses. The trigger attention {βt,j}\{\beta_{t,j}\} makes this independence even clearer: it receives no attribution supervision whatsoever, the sole attribution anchor Lattr\mathcal{L}_{\rm attr} regressing only gˉt\bar{g}_t and never β\beta, yet it localizes the trigger with NDCG 0.830.83 and MAP 0.770.77 against RECCON. The localization is thus emergent and unsupervised, independent of any training objective.


[W4/Q2] The discrete-time baseline is already reported as "-w/o ODE".{\color{#f26921}\text{[W4/Q2] The discrete-time baseline is already reported as "-w/o ODE".}}

We appreciate this comment. The row you cite, removing the ODE costs 2.30, is in fact the discrete-time baseline you ask for: it keeps the self/other split, the CSD drivers, and all training losses, and only replaces the Neural Jump ODE with a direct state update. We apologize that the earlier label "-w/o ODE" did not make this clear. We have relabeled it "-w/o ODE (discrete-time)" in Table 7 and report the construction of every ablation in Appendix 3.6. It scores 70.21 W-F1 on IEMOCAP, 2.30 below the full model, so the continuous-time formulation contributes 2.30 points on top of the decomposition and its auxiliary objectives.


[W5] Multi-seed stability and the best-checkpoint choice.{\color{#f26921}\text{[W5] Multi-seed stability and the best-checkpoint choice.}}

We appreciate this comment. The three-seed results are reported in Appendix 3.3:

Methodseed 22seed 1246seed 3407mean ± std
PSYCHE71.7771.9072.5172.06 ± 0.40
GraphSmile69.6371.0171.3270.65 ± 0.90
ECERC70.1369.1270.2469.83 ± 0.62
EMART68.5267.3768.0267.97 ± 0.58

PSYCHE still leads the strongest baseline GraphSmile by 1.41 points on the three-seed mean, and its standard deviation (0.40) is the smallest, so the lead is stable across seeds.


[W6/Q3] RECCON trigger evaluation: sample count, construction, and independence from training.{\color{#f26921}\text{[W6/Q3] RECCON trigger evaluation: sample count, construction, and independence from training.}}

These background details are now reported in the manuscript and the Appendix.

1. Evaluation subset and scale. The human-verified subset contains n=41n=41 transitions, built from RECCON's 665 annotated emotion-cause transitions by keeping the inter-personal and hybrid ones with at least one candidate utterance and a gold cause inside the candidate window, and then the 41 with ∣O∣≥2|O|\ge 2 candidates. No random sampling is involved: the 41 transitions are the exhaustive set satisfying the criteria above. Because n=41n=41 is small, we supplement it with a large-scale offline-annotated proxy evaluation, covering 182182 IEMOCAP and 141141 MELD valid samples.

2. Stability and significance. Across the three training seeds, the results are STable 5, Appendix, Sec. 3.7:

Seed2212463407mean ± std
NDCG0.81130.83720.82900.826 ± 0.013
MAP0.76770.77490.76830.770 ± 0.004

On the seed-3407 checkpoint we bootstrap the n=41n{=}41 transitions and obtain 95% confidence intervals of NDCG 0.8290.829 [0.775,0.883][0.775, 0.883] and MAP 0.7680.768 [0.695,0.842][0.695, 0.842]. A paired bootstrap against the Uniform baseline gives p=0.0047p=0.0047 (NDCG) and p=0.0034p=0.0034 (MAP).

3. Independence from training. No trigger label or RECCON-derived information influences training: the trigger attention β\beta receives no attribution supervision, and the offline attribution prior only anchors the self-cause ratio gˉ\bar{g}, never β\beta.


We would be grateful if the reviewer would revisit the rating in light of these.

Thank you for the author's response. I am keeping my original rating.

AuthorsOpenReview ↗

Response to Reviewer cNzX

We sincerely thank the reviewer for taking the time to carefully consider our responses. We are delighted that our clarifications and additional experimental results have addressed your concerns and strengthened your confidence in our work. We greatly appreciate your recognition and support.

Reviewer_PpHLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper recasts conversational emotion recognition from per-utterance classification to continuous-time latent trajectory modeling. PSYCHE uses a causal graph encoder to fuse multimodal signals per turn, a Neural Jump ODE to evolve a latent emotional state between utterances, and Causal Source Decomposition (CSD) to split the ODE's driving force into a self-cause driver (state-dependent inertia) and an other-cause driver (interlocutor perturbation). A per-dimension gate fuses the drivers, and a decoupled decoder reads the label from the pre-speech state and the two drivers separately. The self-cause ratio and trigger attention are claimed as forward-pass intrinsic attribution. The paper reports 72.51% weighted F1 on IEMOCAP and 67.55% on MELD, and in Table 3 compares against three LLM-based attribution methods on IEMOCAP only.

Critical strengths
  • Conceptual reframing is substantive. Moving from detecting shifts to evolving a trajectory from which shifts are derived addresses a real limitation of auxiliary shift-detection heads. The CSD ablation, the largest single drop at 4.97 points, confirms that causal decomposition rather than the ODE itself drives the gain.
  • Baseline retraining is fair. All available baselines are retrained from public code on the same splits and features.
Critical weaknesses
  • Selective reporting in the LLM comparison. Table 3 compares against CMTD, PRC-Emo, and Causal-ERC only on IEMOCAP, although all three report MELD results. PRC-Emo reports 70.44 W-F1 and Causal-ERC reports 71.89 on MELD, both above PSYCHE's 67.55. The omitted benchmark is precisely the one where PSYCHE underperforms. This directly undermines the paper's claim of competitive intrinsic attribution relative to LLM-based methods.
  • No comparison with CAS-ODE. CAS-ODE (IEEE SPL, 2026) jointly learns adaptive conversational structures and continuous ODE dynamics on IEMOCAP and MELD, targeting the same discrete-step limitation that motivates PSYCHE. It is not cited or compared, leaving unclear what CSD offers beyond existing ODE-based MERC.
Questions for authors
  • PRC-Emo reports 70.44 and Causal-ERC reports 71.89 weighted F1 on MELD, both above PSYCHE's 67.55. Why are these results excluded from Table 3? Please add the MELD comparison or justify precisely why MELD is not a valid comparison point for these methods.
  • How does PSYCHE's CSD decomposition differ from CAS-ODE's graph ODE formulation, and what does CSD offer that CAS-ODE does not? A conceptual comparison is the minimum expected.
4 comments
AuthorsOpenReview ↗

Response to Reviewer PpHL

We thank the reviewer for the careful reading, the constructive comments, and the recognition of our conceptual reframing and fair baseline retraining, all of which have helped us improve the manuscript.


[W1/Q1] MELD was omitted from the LLM-attribution comparison.{\color{#f26921}\text{[W1/Q1] MELD was omitted from the LLM-attribution comparison.}}

One clarification on the cited numbers: Causal-ERC's 71.89 W-F1 is its IEMOCAP score, not MELD; on MELD it reports 70.25 (p.6 of the original paper)

We apologize for the omission, and we have now added the complete MELD comparison to Table 4 (both WA and W-F1; see the revised manuscript, around L.388):

MethodLLMAttr.IEMOCAP WAIEMOCAP W-F1MELD WAMELD W-F1
CMTD✓✗65.8664.2964.2364.17
PRC-Emo✓✗71.0371.9571.5070.44
Causal-ERC✓✗71.6671.8970.5070.25
PSYCHE✗✓72.5872.5168.6267.55

(Table 4: Comparison with LLM-based attribution methods on IEMOCAP and MELD. LLM: uses a large language model; Attr.: reads attribution directly off the forward pass.)

PSYCHE is best on IEMOCAP and within 3 points of the two strongest LLM methods on MELD. The comparison is not like-for-like: † numbers are taken from the original papers with their own backbones and preprocessing, and these methods require an LLM at inference. Moreover, there are two other reasons for this gap. First, PSYCHE's self/other decomposition separates a speaker's own inertia from the interlocutor's influence, which is most effective on dyadic conversations such as IEMOCAP; in MELD's fast multi-party exchanges, the other-cause driver must aggregate perturbation from several speakers, diluting the signal. Second, these LLM-based methods benefit from open-domain reasoning and handle such complex scenes better.

We have revised the corresponding analysis in the manuscript (around L.389). Two points keep the core contribution intact. First, among the re-evaluated non-LLM baselines, PSYCHE still achieves the best W-F1 on both IEMOCAP and MELD (Table 3). Second, PSYCHE's core contribution is the source-level attribution, not the MELD score itself: the self/other decomposition and the trigger localization are read directly off its forward pass, without any large language model.

Meanwhile, multi-party dialogue is also a genuine need in affective computing. Given PSYCHE's limitation on such scenes, we have listed this as future work in the Conclusion (around L.533) and will continue to explore it.


[W2/Q2] CAS-ODE is missing from the related work.{\color{#f26921}\text{[W2/Q2] CAS-ODE is missing from the related work.}}

Thank you for pointing us to CAS-ODE (Zhang et al., IEEE SPL 2026). We have added it, together with DF-GCN (Meng et al., KBS 2026), to the "Neural ODEs for Emotion Dynamics" paragraph in Related Work (Sec. 2, around L.158).

CAS-ODE and PSYCHE share the motivation of replacing discrete per-utterance steps with continuous ODE dynamics, but they differ in what drives the ODE and what the dynamics expose:

  1. Task objective. CAS-ODE, like DF-GCN and DGODE, treats emotion recognition as its sole objective, whereas PSYCHE is a multi-task model that performs emotion recognition and emotion attribution jointly.

  2. Decomposition of the driver. CAS-ODE jointly learns an adaptive conversational structure and continuous dynamics via a variational hypergraph autoencoder combined with a graph ODE, with the vector field driven by graph-aggregated observations. It does not separate the speaker's own inertia from the interlocutor's influence. CSD, in contrast, explicitly splits the sole control of the Neural Jump ODE into self-cause and other-cause drivers fused by a per-dimension gate.


Our request

Both concerns are addressed in the revised PDF. The reviewer rated soundness as “minor or fixable concerns” and named the conceptual reframing and fair baseline retraining as strengths; we respectfully ask the reviewer to reconsider the rating.

Thanks for the detailed response. The added MELD results, the correction to the Causal-ERC number, and the inclusion of CAS-ODE with a conceptual distinction improve transparency and help clarify how the paper is positioned relative to prior work.

However, the main issues remain. Even with the added MELD comparison, PSYCHE remains below PRC-Emo and Causal-ERC on MELD. The authors explain this by the method’s limitation in multi-party settings and list it as future work, but that does not resolve the original concern about the claim of competitive intrinsic attribution relative to LLM-based methods. On CAS-ODE, the response gives a conceptual distinction, which meets the minimum expectation, but there is still no empirical comparison, so it is hard to tell whether CSD’s interpretability benefits come at a cost in predictive performance. I’m keeping my rating.

AuthorsOpenReview ↗

#2 Response to Reviewer PpHL [1/2]

We thank the reviewer for the careful reading and for the substantive comments. We address the two remaining concerns directly, with new evidence for the attribution quality.


[W1/Q1] The MELD gap and the "competitive" claim.{\color{#f26921}\text{[W1/Q1] The MELD gap and the "competitive" claim.}}

The reviewer notes: "Even with the added MELD comparison, PSYCHE remains below PRC-Emo and Causal-ERC on MELD." We respond in two points:

First, we never claimed to beat all methods on performance. The core contribution of the paper is the conceptual reframing, not performance. As you recognized in your Critical Strengths, "Conceptual reframing is substantive…", our contribution is to reframe emotion from discrete labels into a continuously evolving trajectory, explicitly modeling the two kinds of factors that shape an emotion change through the self/other causal source decomposition. The evidence for this contribution is exactly the CSD ablation you also pointed out: removing CSD brings the largest single drop of 4.97 W-F1, confirming that it is the causal decomposition, not the ODE itself, that drives the gain.

Second, even looking only at performance, PSYCHE is "competitive", consistent with the paper's statement. The table below summarizes PSYCHE's ranking in Table 3 and Table 4:

ComparisonIEMOCAPMELD
Non-LLM baseline (Table 3)1st (72.51)1st (67.55)
LLM methods (Table 4)1st (72.51)3rd (67.55 vs 70.44 / 70.25)

In terms of performance, PSYCHE is first in three of the four cells, and trails only in the LLM comparison on MELD. Therefore, lagging behind LLM-backbone methods on MELD does not constitute a substantive challenge to the paper's claim.


[W1/Q1] The attribution quality relative to an LLM reference.{\color{#f26921}\text{[W1/Q1] The attribution quality relative to an LLM reference.}}

The core of the reviewer's concern is "the claim of competitive intrinsic attribution relative to LLM-based methods". This phrasing merges two independent claims of the paper into one. "Competitive" points to the performance claim, anchored to the non-LLM baselines. "Intrinsic attribution" points to the attribution claim, anchored to the attribution mechanism. We never claimed to be inferior to LLMs in attribution accuracy. On the contrary, our attribution is significantly better in quality than an LLM reference.

On attribution quality, the substantive question is whether PSYCHE's forward-pass attribution is comparable in quality to an LLM-produced attribution, not just cheaper. We run a controlled comparison on RECCON's causal-type labels, using the same subset on which the LLM prior is defined (63 turns: 32 self-contagion, 31 inter-personal):

Signalself-contagion (mean)↑inter-personal (mean)↑AUROC↑
PSYCHE forward-pass (self-cause ratio)0.760.420.82
LLM-generated prior (self/other ratio)0.640.650.48

PSYCHE's forward-pass attribution separates the two causal types with AUROC 0.82, while the LLM prior reaches only 0.48, below chance. (On the full RECCON set, PSYCHE's AUROC is 0.71, as reported in the paper.) Since both are evaluated on the same subset, the comparison is like-for-like. PSYCHE's forward-pass attribution significantly exceeds an LLM-produced reference in accuracy, while requiring no LLM backbone.

AuthorsOpenReview ↗

#2 Response to Reviewer PpHL [2/2]


[W2/Q2] CAS-ODE and the cost of interpretability.{\color{#f26921}\text{[W2/Q2] CAS-ODE and the cost of interpretability.}}

The reviewer notes: "On CAS-ODE, the response gives a conceptual distinction, which meets the minimum expectation, but there is still no empirical comparison, so it is hard to tell whether CSD's interpretability benefits come at a cost in predictive performance." Two points need clarification here:

1. CAS-ODE cannot enter the comparison fairly, and importing its reported numbers would be even less fair.

Regarding CAS-ODE (G. Zhang, Y. Zhang, L. Wang, J. Zhang, and X. Cai, "CAS-ODE: Jointly Learning Adaptive Structures and Continuous Dynamics for Emotion Recognition in Conversation," IEEE Signal Processing Letters, vol. 33, pp. 2405\–2409, 2026, doi: 10.1109/LSP.2026.3699337) , whether it can settle the question of CSD depends on whether it can enter a fair comparison. As you recognized in your Critical Strengths: "Baseline retraining is fair. All available baselines are retrained from public code on the same splits and features." All comparison models in Table 3 are retrained from public code under the same splits and features. CAS-ODE provides no public code and no experimental parameters, so it cannot be retrained under the same standard. Directly importing its reported numbers into the comparison violates the fairness standard you endorsed.

2. Whether CSD costs performance does not require CAS-ODE to judge. The CSD ablation answers it directly.

The standard way to judge whether a module costs performance is to ablate it: if it costs performance, removing it should raise performance. Our CSD ablation (Table 5) shows that removing CSD drops W-F1 from 72.51 to 67.54 (−4.97, the largest single drop in the table). In other words, CSD is the largest positive contributor to predictive performance, not a cost. Therefore, CSD's interpretability benefits and predictive performance are not zero-sum. It provides both at the same time.


We would be grateful if the reviewer would revisit the rating in light of these.

Reviewer_yeTiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a new conversational emotion recognition method (PSYCHE) that models each speaker's emotion as a continuously evolving latent trajectory rather than a sequence of independent utterance-level labels. In the proposed method, a causal graph encoder is first used to partition the historical context into two information streams: same-speaker and other-speaker. Then, Causal Source Decomposition (CSD) is utilized to construct a self driver based on emotional inertia and an other-speaker driver based on attention, respectively. These are then fused via a per-dimension gate and finally serve as the control signal for Neural Jump ODE. Additionally, the model outputs a self-cause ratio and trigger attention, aiming to obtain attribution directly from the forward computation. Experiments on IEMOCAP and MELD are conducted to validate the performance of the proposed method.

Critical strengths

This work remodels conversational emotion as a latent dynamical process, rather than simply adding fusion or contextual modules, with technical soundness. The proposed method presents a relatively comprehensive and meaningful modeling approach, by continuous evolution, speech-act jumps, and the explicit separation of self-history from interlocutor influence. The motivation of this work is clear and the experimental results also validate the effectiveness of the proposed method.

Critical weaknesses
  1. In the proposed method, the decomposition is constructed based on speaker identity and temporal ordering, which achieves source separation but does not automatically constitute causal identification. Whereas, this paper did not present enough interventions, counterfactual tests, or explicit causal assumptions, to prove that Φ^self and Ψ^other can be interpreted as true causal effects. Moreover, the gate is not an entirely spontaneously emerging attribution variable: L_gate explicitly requires a lower self-cause ratio during emotion shifts, while L_attr directly regresses the gate mean toward the LLM-generated self/other attribution prior. It implies that the observed self-cause ratio decreases during shifts may be caused by the training targets themselves. It is inadequate that claims regarding "causal source decomposition" and "intrinsic attribution".

  2. Although this paper retrains most baselines using the same multimodal features, PSYCHE uses additional offline LLM-generated entity-level and scene-level commonsense cues, which means PSYCHE receives additional information compared to other methods. Furthermore, the ablation study shows that IEMOCAP W-F1 drops from 72.51 to 70.74 after removing all commonsense cues, which is lower than the 71.32 reported by GraphSmile. Maybe, the current headline SOTA result cannot disentangle the contribution of the continuous trajectory/CSD itself from the gains of the additional LLM-generated knowledge. It is better to provide a controlled comparison under identical input information.

Questions for authors
  1. It is difficult to understand why Φtself\Phi_t^{self} and Ψtother\Psi_t^{other} can be interpreted as causal effects, rather than merely source-conditioned predictive representations, and it is based what causal assumptions. Could this causal interpretation be validated through interventions or counterfactual experiments?

  2. Since L_gate explicitly requires g ̅_shift<g ̅_stay, how should the difference in self-cause ratio between shift and stay be viewed in experiments as an independent validation, rather than merely a direct result of the training objective?

  3. Given that current ablation experiments demonstrate that these additional cues make a substantial contribution to the SOTA results, could the authors provide a controlled SOTA comparison without using additional LLM-generated commonsense cues, or grant competing methods access to the same input information, in order to standardize the evaluation?

  4. The current time interval is defined as δ=(t-t_prev )/T, where t is the utterance index rather than a real timestamp. How should the paper's discussion on temporal decay and silent intervals be understood in light of this? If timestamps are available in the data, have the authors tried using real elapsed time?

  5. What is the exact number of samples in the human-verified subset used for trigger localization, and what sampling method was employed? It is recommended to explicitly report confidence intervals or significance tests for NDCG/MAP.

3 comments
AuthorsOpenReview ↗

Response to Reviewer yeTi [1/2]

We thank the reviewer for the careful reading of our paper and for the detailed and insightful comments!


${\color{#f26921}\text{[W1/Q1] Interpretation of Φtself\Phi^{\rm self}_t and Ψtother\Psi^{\rm other}_t as causal.}}$

Our claim is source attribution under one explicit assumption, time-admissibility (Sec. 3.2), not interventional identification. “Causal” follows the emotion-cause usage of RECCON [1]^{[1]}: attributing an emotion change to its source, the speaker’s own prior state or an interlocutor’s utterance. Φtself\Phi^{\rm self}_t and Φtother\Phi^{\rm other}_t are source-conditioned drivers of the ODE, not identified causal effects, and the revision removes every phrase that invited the stronger reading ("intrinsic" → "readable"). The evidence that matches this claim is agreement with independent human cause annotations (RECCON, Q5) and the counter-ablation below (Q2).


[W1/Q2] Independence of the shift-vs-stay ordering in the self-cause ratio.{\color{#f26921}\text{[W1/Q2] Independence of the shift-vs-stay ordering in the self-cause ratio.}}

The ordering is not produced by Lgate\mathcal{L}_{\rm gate}. Removing both terms that act on Lgate\mathcal{L}_{\rm gate} ( {L}attr\mathcal\{L\}_{\rm attr} and Lgate\mathcal{L}_{\rm gate} ) leaves 92% of the shift–stay gap: Δ = 0.154 (0.386 vs. 0.540, p = 6.5×10−24) versus Δ = 0.167 for the full model (Figure 6; see the revised manuscript, around L.504).


${\color{#f26921}\text{[Q4] Time interval δ\delta based on the utterance index.}}$

We clarify that tt here is the utterance index and TT the conversation length, so δ=(t−tprev)/T\delta=(t-t_{\rm prev})/T is the normalized index distance. IEMOCAP transcripts and MELD do provide per-utterance start/end times; our runs follow the standard preprocessed feature release (COGMEN [2]^{[2]} / GraphSmile [3]^{[3]} splits), which does not carry them, so δ is the normalized turn distance. The inaccurate statement in Sec. 3.3.1 has been corrected. Accordingly, "interval" in Sec. 1 and Sec. 3.3.1 now explicitly denotes turn distance, and the decay exp(−γtδ)exp(−γ_{t} \delta) is described as attenuation over intervening turns, not wall-clock silence.


References

[1] Soujanya Poria, Navonil Majumder, Devamanyu Hazarika, Deepanway Ghosal, Rishabh Bhardwaj, Samson Yu Bai Jian, Pengfei Hong, Romila Ghosh, Abhinaba Roy, Niyati Chhaya, et al. Recognizing emotion cause in conversations. Cognitive Computation, 13(5):1317–1332, 2021.

[2] Abhinav Joshi, Ashwani Bhat, Ayush Jain, Atin Singh, and Ashutosh Modi. Cogmen: Contex-tualized gnn based multimodal emotion recognition. In Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies, pp. 4148–4164, 2022.

[3] Jiang Li, Xiaoping Wang, and Zhigang Zeng. Tracing intricate cues in dialogue: Joint graph structure and sentiment dynamics for multimodal emotion recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.

AuthorsOpenReview ↗

Response to Reviewer yeTi [2/2]

[W2/Q3] Fairness of the SOTA comparison with the LLM commonsense cues.{\color{#f26921}\text{[W2/Q3] Fairness of the SOTA comparison with the LLM commonsense cues.}}

We run two experiments to isolate the architecture's contribution from the cues.

1. Granting the strongest baselines the same cues. We retrain GraphSmile and ECERC with the same commonsense cues, as reported in STable 5, Appendix, Sec. 3.7:

Methodsw/o com.w/ com.Δ
GraphSmile71.3272.28+0.96
ECERC70.2470.45+0.21
PSYCHE70.7472.51+1.77

(EMOCAP W-F1 %. com. = commonsense cues.)

With the cues, PSYCHE and GraphSmile are on par (72.51 vs. 72.28); without them, PSYCHE is 0.58 below GraphSmile (70.74 vs. 71.32). The cues help GraphSmile by +0.96 and leave ECERC nearly unchanged (+0.21), so most of the 1.19-point lead comes from the cues. We therefore do not claim it as an accuracy lead — the abstract says "competitive".

2. The CSD gain is independent of the cues. To further verify that the architecture still contributes without the cues, we isolate CSD. A 2$\times$2 cross-ablation that varies the commonsense cues and CSD isolates the CSD contribution, as reported in Table 6, Sec. 4.3, around L.544:

Settingsw/ CSDw/o CSDCSD gain
w/ commonsense72.5167.54+4.97
w/o commonsense70.7466.33+4.41

(Table 6: IEMOCAP W-F1.)

CSD adds +4.97 with the cues and +4.41 without them, so the CSD contribution does not depend on the cues. The cues capture what the conversation is about; CSD captures how and why the emotion changes.

3. The core contribution is the attribution, not the accuracy. At the same accuracy as the strongest baseline, PSYCHE additionally yields two things no baseline provides: how much of an emotion change comes from the speaker's own state (the self-cause ratio), and which utterance triggered the change (the trigger-localizing attention). Both are read directly off the forward pass that also predicts the label, without a separate explainer or an inference-time LLM. Both match the emotion-cause annotations of RECCON, an independent human-labeled dataset. This attribution, available for free at inference, is our core contribution.


[Q5] Sample size and significance for trigger localization.{\color{#f26921}\text{[Q5] Sample size and significance for trigger localization.}}

The human-verified subset contains n=41n=41 transitions. Starting from RECCON's 665 annotated emotion-cause transitions, we keep the inter-personal and hybrid ones, the two types that carry an interlocutor trigger to localize, with at least one candidate utterance and a gold cause inside the candidate window, leaving 116. Among these we keep the 41 with ∣O∣≥2|O|\ge 2 candidates, because a single candidate is selected trivially by any method and offers no discriminative power. No random sampling is involved: the 41 transitions are the exhaustive set satisfying the criteria above.

Because n=41n=41 is small, we supplement it with two measures:

  1. Larger-scale validation: a large-scale offline-annotated proxy evaluation covering 182182 IEMOCAP / 141141 MELD valid samples, where β\beta again outperforms the Uniform baseline across all five ranking metrics.

  2. Uncertainty quantification: bootstrap 95% confidence intervals give NDCG 0.8290.829 [0.775,0.883][0.775, 0.883] and MAP 0.7680.768 [0.695,0.842][0.695, 0.842], and both lower bounds stay above the Uniform baseline's NDCG 0.7570.757 and MAP 0.6700.670. A paired bootstrap test over the same n=41n=41 transitions yields p=0.0047p=0.0047 for NDCG and p=0.0034p=0.0034 for MAP.

Full statistics are in Appendix 3.4.


In response to the readability concern, we have also substantially revised Section 3 (Method). Guided by your questions on the causal terminology (Q1), attribution independence (Q2), and the time interval (Q4), this section is now easier to follow. We would be grateful if the reviewer would revisit the rating in light of these.

Dear Reviewer,

Thank you for your thoughtful comments and questions. We would like to kindly follow up on our rebuttal, as the discussion period is approaching its end. We hope that our detailed responses and revisions have adequately addressed your concerns.

We sincerely appreciate your time and consideration, and would be grateful if you could revisit our responses before the discussion period closes.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their constructive feedback and thoughtful engagement with our work. We are grateful for their recognition of the paper's key strengths, and their suggestions have helped us further improve the manuscript.

Strengths. Reviewers recognized the conceptual reformulation of conversational emotion recognition as a substantive contribution [PpHL, yeTi, cNzX]. Instead of detecting emotion shifts with auxiliary heads, PSYCHE models each speaker's emotion as a continuously evolving latent trajectory, from which shifts are derived. They also highlighted the causal source decomposition, which makes the speaker's own inertia and the interlocutor's influence visible as two explicit sources [cNzX, PpHL], the fair retraining of all baselines [PpHL], and the comprehensive evaluation that combines ablation with dynamics analysis [cNzX, yeTi].

In response to the reviewers' concerns, which center on the empirical evaluation, we have strengthened the paper in three directions:

  1. Fairness of the comparisons. We added controlled comparisons that grant the strongest baselines the same offline commonsense cues, completed the MELD comparison against the LLM-based methods, and positioned the related ODE baseline CAS-ODE. Evidence and scope of the attribution. We added an ablation that removes the attribution losses to test the independence of the gate, reported the significance and sample construction for trigger localization, and clarified the scope of "causal" as source-level decomposition.
  2. Reproducibility and disclosure. We disclosed the prompts, visible context, and filtering for the offline LLM annotations, together with multi-seed results and significance tests. These additions strengthen the empirical evaluation without changing the core method. We have uploaded a revised manuscript and a revised appendix reflecting these changes.

We hope the revised manuscript and our detailed responses adequately address the reviewers' concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.