- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EmoInsight, a framework for fine-grained visual emotion assessment, along with ContiEmo, a new dataset containing 105K images with dense affective variations and human annotations. EmoInsight combines preference-guided multi-teacher distillation to enhance affective reasoning with coarse-to-fine factor-guided reinforcement learning (FED-GRPO) to improve valence-arousal estimation. Experiments on three datasets demonstrate improvements over existing methods.
Critical strengths
- ContiEmo introduces densely sampled homologous images with rich human annotations, providing valuable supervision for fine-grained visual emotion assessment.
- The combination of preference-guided multi-teacher distillation and factor-guided reinforcement learning achieves consistent improvements across multiple benchmarks, supported by extensive experiments and ablation studies.
Critical weaknesses
- The construction of the dataset requires more theoretical and effective support:
- The four human annotation tasks are intuitively reasonable, but their theoretical justification and annotation reliability remain insufficiently validated. In particular, what motivates the selection of the seven coarse-grained affective factors and the three fine-grained factors?
- Further evaluation of the visual quality and semantic consistency of generated intermediate frames would strengthen the reliability of ContiEmo.
- Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.
Questions for authors
-
The four human annotation tasks are intuitively reasonable, but their theoretical justification and annotation reliability remain insufficiently validated. In particular, what motivates the selection of the seven coarse-grained affective factors and the three fine-grained factors?
-
Further evaluation of the visual quality and semantic consistency of generated intermediate frames would strengthen the reliability of ContiEmo.
-
Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.
-
Comparisons with top-ranked CoT selection and learnable adaptive weighting would better validate the effectiveness of multi-teacher distillation.
4 comments
Response to Reviewer tqTf on More Evaluations and Comparisons. (2/2)
Comment 3:
Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.
Authors’ Response:
Thank you for this comment. Our original ContiEmo test predictions already include homologous images. We conduct an additional pairwise analysis to assess the model's ability to capture fine-grained affective variations.
For each pair of images originating from the same base image, we compare the predicted and human-annotated score differences along Valence and Arousal independently. A prediction is considered correct if both differences have the same sign. We use group-balanced Pairwise Accuracy (PAcc) to ensure that each base-image group contributes equally to the overall evaluation.
-
Overall pairwise discrimination. Table R3-4 shows that EmoInsight achieves 77.3% Valence PAcc and 71.6% Arousal PAcc, surpassing all competing methods.
-
Sensitivity to affective difference magnitude. We further categorize image pairs into small (0 < |Δy| ≤ 0.5), moderate (0.5 < |Δy| ≤ 1.0), and large (|Δy| > 1.0) groups. As shown in Table R3-5, EmoInsight obtains better performance than the strongest baseline Qwen3-VL-8B across all three groups, demonstrating its ability to distinguish even subtle affective variations.
The pairwise results provide evidence that EmoInsight captures fine-grained affective variations within homologous images. We have incorporated the evaluation protocol and findings into Section 4.4.3 [Effectiveness of Dense Homologous Frames] of the revised manuscript.
Table R3-4: Overall Pairwise Ordering Accuracy.
| Method | V-PAcc ↑ | A-PAcc ↑ |
|---|---|---|
| StyleEDL | 60.6 | 54.8 |
| EmoSet | 68.0 | 61.7 |
| VA-CLIP | 62.9 | 57.4 |
| MFRN | 66.4 | 62.3 |
| Qwen3-VL-4B | 64.8 | 60.0 |
| Qwen3-VL-8B | 68.7 | 63.5 |
| EmoInsight | 77.3 | 71.6 |
Table R3-5: Ordering Accuracy under Different Affective Differences.
| Method | V-Small ↑ | V-Moderate ↑ | V-Large ↑ | A-Small ↑ | A-Moderate ↑ | A-Large ↑ |
|---|---|---|---|---|---|---|
| Strongest Baseline | 60.2 | 67.5 | 77.6 | 56.0 | 63.1 | 74.3 |
| EmoInsight | 70.4 | 76.1 | 83.2 | 65.8 | 72.0 | 80.1 |
Comment 4:
Comparisons with top-ranked CoT selection and learnable adaptive weighting would better validate the effectiveness of multi-teacher distillation.
Authors’ Response:
Thank you for this comment. We have extended the multi-teacher distillation ablation with two comparisons:
- Top-ranked CoT Selection, which retains only the highest-ranked teacher CoT for each image based on human preferences, equivalent to a one-hot teacher weighting.
- Learnable Adaptive Weighting, which employs a two-layer MLP (Linear(d, 256)-GELU-Linear(256, 3)) to predict image-conditioned teacher weights from mean-pooled visual features. The MLP is trained using the human preference distributions with a KL-divergence loss, and is then used to generate teacher weights for student SFT.
As shown in Table R3-6, our Dynamic Soft Weighting outperforms both Top-ranked CoT Selection and Learnable Adaptive Weighting, demonstrating that retaining the full human preference distribution is more effective than hard teacher selection or learnable weighting in multi-teacher distillation.
Table R3-6: Comparisons with Top-ranked CoT Selection and Learnable Adaptive Weighting.
| Method | V-Err ↓ | A-Err ↓ |
|---|---|---|
| Uniform Weighting | 0.711 | 0.349 |
| Global Fixed Weighting | 0.701 | 0.327 |
| Top-ranked CoT Selection | 0.628 | 0.321 |
| Learnable Adaptive Weighting | 0.623 | 0.316 |
| Ours: Dynamic Soft Weighting | 0.614 | 0.309 |
We have supplemented the additional multi-teacher distillation ablation in Section 4.4.1 [Effectiveness of Multi-Teacher Synergistic Affective Reasoning mechanism] of the revised manuscript.
We have conducted extensive evaluations and analyses to address all the concerns, and sincerely hope the reviewer can reconsider the overall score based on these revisions.
Official Comment by Authors
Dear Reviewer tqTf,
We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.
Could you please kindly let us know if our rebuttal has addressed your concerns?
If you have any further questions regarding our work, we would be delighted to address them.
If our responses have addressed your concerns, we would greatly appreciate your reconsideration of your overall assessment.
Best regards,
Authors
Dear Reviewer tqTf,
As the rebuttal deadline is just over 24 hours away, we would greatly appreciate any feedback on our responses.
Thank you!
Response to Reviewer tqTf on More Evaluations and Comparisons. (1/2)
We sincerely appreciate your recognition of our dataset contribution and method effectiveness. Below, we address your constructive comments with detailed point-by-point responses and relevant revisions in the paper.
Comment 1:
Authors’ Response:
We thank the reviewer for the valuable comment.
Theoretical justification for factor selection. We organize the seven coarse-grained affective factors into three levels according to their visual hierarchy and semantic roles in affective perception: | Level | Factors | Visual Information | |:-|:-|:-| | Low-level Visual Cues | Weather, Lighting, Color | Environmental conditions and appearance characteristics | | Mid-level Structural Cues | Composition, Perspective | Spatial layout, viewpoint, and visual organization | | High-level Semantic Cues | Elements, Theme | Scene entities, events, and semantic context |
Annotation Reliability. We further assess the inter-annotator agreement of ContiEmo through statistical analyses.
As shown in Table R3-1, five-rater aggregated V-A ratings demonstrate higher reliability than individual ratings, while the factor rankings and teacher preferences exhibit relatively high inter-annotator agreement. These analyses support the reliability of ContiEmo annotations.
We have added the theoretical justification and reliability results to Section 4.5 [Annotation Reliability and Justification of ContiEmo] of the revised manuscript.
Table R3-1: Inter-Annotator Reliability of ContiEmo Annotations.
Comment 2:
Authors’ Response:
Thank you for this suggestion. We conduct a human evaluation of the intermediate frames.
Specifically, we randomly sample 500 base-image–sequence pairs, with balanced coverage of the four editing directions. Each sequence is independently evaluated by three annotators on a 5-point scale in two aspects:
We report the mean and std of the ratings, with inter-annotator agreement measured by ICC(2,1) and ICC(2,3). The results are presented in Table R3-2 and Table R3-3. The generated sequences achieve consistently high ratings in both semantic consistency and visual quality across all four editing directions, with good inter-annotator agreement.
Table R3-2: Human evaluation of intermediate-frame quality.
Table R3-3: Inter-annotator Agreement.
We have added this evaluation to Appendix C.3 [Human Evaluation of the Intermediate Frames].