- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces a perception-to-appraisal paradigm for MERg, in which observable evidence is interpreted through six cognitive appraisal dimensions before predicting an emotion. It contributes CogEmo-40K for instruction tuning, CogEmo-Bench with AEQS for appraisal-oriented evaluation, and CogEmo-MoE, a compact model using interleaved sparse experts and progressive training. Experiments on the proposed benchmark and multiple external emotion datasets demonstrate improvements in both appraisal evidence quality and emotion recognition.
Critical strengths
-
The motivation is reasonable, and the overall contribution is relatively solid, covering a training dataset, an appraisal-oriented evaluation benchmark, and a corresponding model.
-
The paper is generally well presented, with a clear progression from motivation to data construction, model design, and evaluation. The experiments are also fairly extensive, including comparisons with diverse MLLMs, architectural ablations, modality analysis, and cross-domain evaluation.
Critical weaknesses
-
My main concern is how appraisal is operationally framed for both training and evaluation. Although the main text names the six appraisal dimensions and outlines the construction pipeline, their definitions and demonstrations are absent (from the main text). These choices are central to the paper because CogEmo-40K and CogEmo-Bench use the same appraisal framework and closely aligned construction protocols. It is therefore difficult to determine whether the model learns deeper cognitive appraisal or primarily learns to reproduce the expected dimension-specific language and emotion-correlated appraisal patterns.
-
(Minor) The technical contribution of CogEmo-MoE appears relatively limited. The method largely builds on MoE-LLaVA by copying FFN weights into sparse experts and applying progressive supervised fine-tuning, making it closer to an architectural and training adaptation.
Questions for authors
- Could the authors summarize more concretely what constitutes each appraisal dimension in the training targets and evaluation references, and clarify how closely the two construction procedures are aligned? More importantly, what evidence demonstrates that the model performs evidence-grounded appraisal reasoning rather than imitating recurring appraisal phrases or emotion-specific templates learned from CogEmo-40K?
- Could the authors perform some RL-based optimization on baselines and analyze how they contribute to this new reasoning paradigm?
4 comments
[2/3] Response to Reviewer M11W
Weakness 2:Is the technical contribution of CogEmo-MoE limited to an architectural and training adaptation of MoE-LLaVA?
Response. Thank you for this observation. We agree that the MoE architecture itself is not intended to be a fundamentally new MoE design. CogEmo-MoE builds on the MoE-LLaVA framework, while our main methodological focus is on adapting sparse conditional computation to the proposed Perception → Appraisal → Emotion paradigm through progressive multimodal alignment, perception enhancement, and cognitive appraisal tuning.
Our additional controlled ablation further clarifies this distinction. Specifically, we conducted a 2 × 2 ablation while keeping Stages 1 and 2 fixed and initializing all models from the same Stage-2 checkpoint. We varied only two factors in Stage 3: (1) the supervision target, comparing Emotion Only with Appraisal + Emotion, and (2) the architecture, comparing Dense with 4-expert Top-2 MoE. The source samples, training budget, and hyperparameters were otherwise kept the same.
| Stage-3 Supervision | Architecture | AEQS ↑ | Emotion Acc. ↑ | Macro-F1 ↑ |
|---|---|---|---|---|
| Emotion Only | Dense | 2.10 | 66.74 | 53.60 |
| Emotion Only | 4E Top-2 MoE | 1.98 | 68.35 | 53.04 |
| Appraisal + Emotion | Dense | 70.48 | 67.26 | 53.27 |
| Appraisal + Emotion | 4E Top-2 MoE | 70.60 | 71.43 | 57.17 |
The large AEQS increase from Emotion Only to Appraisal + Emotion shows that cognitive appraisal capability is primarily established by appraisal-guided supervision. In contrast, under the same Appraisal + Emotion supervision, replacing Dense with 4E Top-2 MoE changes AEQS only from 70.48 to 70.60 (+0.12), while improving emotion Accuracy from 67.26% to 71.43% (+4.17) and Macro-F1 from 53.27% to 57.17% (+3.90). This indicates that the sparse MoE provides a complementary architectural benefit, particularly for final emotion discrimination, rather than being the primary source of cognitive appraisal reasoning.
We therefore agree that our original wording may overemphasize MoE specialization. In the revision, we will narrow this claim and more clearly position CogEmo-MoE as an architectural realization of the proposed cognitive appraisal reasoning framework, rather than presenting the MoE mechanism itself as the primary technical novelty.
[3/3] Response to Reviewer M11W
Question 2 :Could RL-based optimization of the baselines further contribute to the proposed reasoning paradigm?
Response. Thank you for this valuable suggestion. We agree that reinforcement learning (RL) is a promising direction for further improving cognitive appraisal reasoning. However, we did not introduce RL optimization into the current comparison for two main reasons.
First, adding RL to the baselines would simultaneously introduce additional factors such as the reward design, optimization objective, and training budget, making it more difficult to isolate the contribution of cognitive appraisal supervision itself. Our current controlled experiments therefore focus on separating the effects of appraisal supervision and model architecture under comparable training conditions.
Second, the choice of reward is non-trivial for this task. Using final emotion correctness as the reward would primarily optimize emotion classification rather than appraisal reasoning, whereas directly using AEQS as both the RL reward and the evaluation metric could encourage optimization toward the evaluation metric without independently establishing better grounding. A more principled RL formulation would require an independent grounding-aware reward or human-preference signal that evaluates whether the generated appraisal is genuinely supported by the multimodal evidence.
Beyond correct emotion prediction, our aim is to make the intermediate reasoning more interpretable and better aligned with human appraisal of multimodal evidence. Therefore, in the current work, we focus on establishing and validating the Perception → Appraisal → Emotion paradigm through controlled supervision and grounding-oriented evaluation. We view RL with independent grounding-aware or human-preference rewards as an important extension of this framework and will include this direction in our discussion of future work.
Thank you for the thoughtful and detailed response, which addresses most of my concerns. I will therefore maintain my current score, which is already positive.
[1/3] Response to Reviewer M11W
Response. Thank you for this important comment. Detailed definitions of the six appraisal dimensions are provided in Appendix Table 7, with cross-model examples in Appendix E.4. Their operational meanings are summarized below. The generation protocol and human verification procedure are detailed in Appendices B.2 and B.3, respectively. We will add this summary and clearer appendix pointers to the main text.
For your convenience, we have copied Table 7 below.
CogEmo-40K and CogEmo-Bench share the same cognitive appraisal framework and description-level perception principles to ensure consistency between training and evaluation objectives, but they differ in the strength of quality control. CogEmo-40K adopts a human-in-the-loop verification and correction process, whereas CogEmo-Bench uses held-out samples not involved in Stage-3 training, and its evaluation references are further reviewed and manually refined at the sample level by researchers with prior experience in affective computing. Therefore, CogEmo-Bench does not simply reuse the training annotations.
We also conducted a grounding stress test to examine whether AEQS primarily rewards recurring appraisal phrases or emotion-specific templates. For each test sample, we constructed four candidate explanations: Matched, the appraisal corresponding to the current sample; Paraphrased Matched, a meaning-preserving paraphrase of the matched appraisal with different surface wording; Same-emotion Mismatch, an appraisal transferred from another sample with the same emotion label but different event and appraisal content; and Random Mismatch, an appraisal transferred from an unrelated sample.
The results show that AEQS remains relatively high for Paraphrased Matched explanations despite substantial changes in surface wording, while it drops sharply for Same-emotion Mismatch and Random Mismatch. In particular, the 24.92-point drop under Same-emotion Mismatch, where the emotion label is unchanged, suggests that the evaluation is sensitive to instance-specific appraisal content and context rather than mainly rewarding emotion-specific templates or fixed phrasing.
Taken together, these results show that AEQS is sensitive to instance-specific appraisal content beyond recurring surface phrases. Evidence concerning the learned model is complementary: under Appraisal + Emotion supervision, CogEmo-MoE improves Accuracy from 68.35% to 71.43% and Macro-F1 from 53.04% to 57.17% relative to Emotion-Only training; the cross-domain evaluation in Appendix E.2 further supports the utility of cognition-grounded supervision beyond CogEmo-Bench. These results support the proposed training paradigm. We will make the operational definitions and these distinctions more explicit in the revision.