- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DSPO, a GRPO-based RL framework for emotion reasoning in MLLMs. It addresses visual hallucination and sparse hard-label rewards with two modules. The first is CVIG, which masks the model-predicted evidence region to reward visual grounding and weight candidate emotions. The second is DEDR, which rewards alignment between K candidate emotions and a context-grounded VAD prior. Experiments on EmoSet, Emotion6 and WebEmo with Qwen2.5-VL-3B report gains over GRPO, DAPO and EMO-R3, particularly in out-of-domain accuracy, supported by distribution-based metrics and ablations.
Critical strengths
** Consistent out-of-domain gains.** DSPO improves out-of-domain accuracy over GRPO, DAPO and EMO-R3 under both training protocols and both rollout settings (e.g., 59.38 vs. 53.60 for EMO-R3 at G=8). Robustness across datasets is the key challenge in emotion recognition, so this is the main empirical point in the paper's favor. However, its interpretation is limited by the attribution concerns raised in W7 and Q1.
Reasonably thorough ablations at no inference cost. The paper ablates the two modules, the number of candidates, the diversity formulation and the CVIG design choices. In addition, both modules are used only during training, so inference cost is unchanged. This makes the approach practical and offers some insight into which components drive the gains.
Critical weaknesses
Motivation and claims
W1. Key motivating claims lack empirical support. The paper's motivation rests on three claims: that visual hallucination is "exacerbated" in emotion reasoning, that hard-label rewards cause mode collapse, and that GRPO-style methods "suffer a severe failure." None of these is directly supported. No hallucination metric or training-time diversity statistic is reported. The only evidence is a single case (Fig. 4), in which the "hallucinated" dark clouds are actually present in the image. Moreover, Table 1 shows that GRPO substantially outperforms both zero-shot and SFT, which contradicts the claim of severe failure.
W2. Inconsistency between motivation and task formulation. The method is motivated by the subjective and continuous nature of emotions. Yet the model still outputs a single label, evaluation remains hard-matching accuracy, and the accuracy reward dominates the objective (λ_acc = 1.0 vs. λ_div = 0.3).
W3. Insufficient discussion of related work. The paper does not discuss label distribution learning for visual emotion, nor counterfactual or perturbation-based hallucination mitigation (e.g., visual contrastive decoding). Both are closely related to the proposed modules.
Method
W4. CVIG does not verify what it claims to verify. CVIG only checks whether masking the predicted box changes the candidate probabilities, not whether the textual evidence matches the boxed content. Since both the box and the probabilities come from the same policy model, this is not external verification. The paper’s own criticism of EMO-R3 applies equally here.
W5. DEDR performs first-moment matching rather than distribution alignment. The reward compares only the candidate centroid with a prior mean centered on the ground-truth label, and it never considers the spread of the candidates. In Table 5, Center Distance alone (61.69) barely differs from w/o DEDR (61.29), which suggests that prior alignment contributes little.
W6. The two modules are mechanistically in tension. R_cf rewards a single box that all K candidates depend on, whereas DEDR encourages diverse interpretations that often arise from different regions. As a result, CVIG down-weights plausible interpretations grounded elsewhere; “sadness” in Fig. 4 is an example.
Experiments
W7. Unclear whether baselines use the same prompting template. The first row of Table 3 is identical to GRPO (G=8) in Table 1. Consequently, the effect of the 4-step template is either never validated or mixed into the module contributions. Please specify each baseline’s template and report GRPO with and without it.
Questions for authors
Q1. Handling of label spaces
- Which label set is used for each dataset? For example, is "neutral" removed from Emotion6, and which level of the WebEmo hierarchy is used?
- When testing on a taxonomy the model was not trained on (e.g., trained on EmoSet's 8 classes, tested on Emotion6's 6 classes), how is the prediction made and evaluated?
- Could the out-of-domain gains come from the VAD lexicon mapping between label vocabularies, rather than from modeling subjective diversity? Is there a control that can rule this out?
Q2. Prior construction under the Emotion6 training protocol
- When training on Emotion6, is the proposed lexicon + caption prior replaced by the ground-truth multi-annotator distribution? If so, does this give DSPO extra supervision that the baselines do not have?
- How is the 6-class annotator distribution converted into the VAD Gaussian parameters (μ*, σ*)?
- What are the Emotion6-training results when the proposed lexicon + caption prior is used instead?