acceptodds
Withdrawn from ICLR 2027

EmoInsight: Learning Fine-Grained Visual Affect from Homologous Variations and Preference-Guided Semantic Supervision

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual Emotion Assessment (VEA) aims to model human affective perception elicited by visual content. In real-world scenarios, emotion perception is inherently subjective, arises from reasoning over multiple interacting visual cues, and can vary substantially with subtle changes in visual context. However, existing VEA methods largely treat emotion assessment as numerical scoring. This limits their capacity for deep semantic reasoning over diverse cues and capturing fine-grained visual-affective correlations. To bridge this gap, we introduce a new benchmark dataset (**ContiEmo**) constructed from homologous image groups with densely annotated affective variations and human-annotated multi-cue reasoning labels. Building upon ContiEmo, we develop **EmoInsight**, a preference-guided VEA model. Specifically, a *Multi-Teacher Synergistic Affective Reasoning* mechanism is proposed to enhance semantic reasoning capabilities by transferring diverse affective knowledge with instance-specific dynamic weighting derived from human preferences. To further capture fine-grained visual-affective correlations, we introduce a *Coarse-to-Fine Factor-Guided Reinforcement* strategy that jointly optimizes V-A estimation and consistency with affective factor cues. Extensive experiments on ContiEmo and public real-world VEA benchmarks demonstrate that our method consistently improves fine-grained affect estimation and multi-factor recognition.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Visual Emotion Assessment, Knowledge Distillation, Reinforcement Learning, Fine-Grained Affect Estimation, Homologous Affective Variations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tqTfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EmoInsight, a framework for fine-grained visual emotion assessment, along with ContiEmo, a new dataset containing 105K images with dense affective variations and human annotations. EmoInsight combines preference-guided multi-teacher distillation to enhance affective reasoning with coarse-to-fine factor-guided reinforcement learning (FED-GRPO) to improve valence-arousal estimation. Experiments on three datasets demonstrate improvements over existing methods.

Critical strengths
  1. ContiEmo introduces densely sampled homologous images with rich human annotations, providing valuable supervision for fine-grained visual emotion assessment.
  2. The combination of preference-guided multi-teacher distillation and factor-guided reinforcement learning achieves consistent improvements across multiple benchmarks, supported by extensive experiments and ablation studies.
Critical weaknesses
  1. The construction of the dataset requires more theoretical and effective support:
  • The four human annotation tasks are intuitively reasonable, but their theoretical justification and annotation reliability remain insufficiently validated. In particular, what motivates the selection of the seven coarse-grained affective factors and the three fine-grained factors?
  • Further evaluation of the visual quality and semantic consistency of generated intermediate frames would strengthen the reliability of ContiEmo.
  • Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.
Questions for authors
  1. The four human annotation tasks are intuitively reasonable, but their theoretical justification and annotation reliability remain insufficiently validated. In particular, what motivates the selection of the seven coarse-grained affective factors and the three fine-grained factors?

  2. Further evaluation of the visual quality and semantic consistency of generated intermediate frames would strengthen the reliability of ContiEmo.

  3. Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.

  4. Comparisons with top-ranked CoT selection and learnable adaptive weighting would better validate the effectiveness of multi-teacher distillation.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tqTf on More Evaluations and Comparisons. (1/2)

We sincerely appreciate your recognition of our dataset contribution and method effectiveness. Below, we address your constructive comments with detailed point-by-point responses and relevant revisions in the paper.

Comment 1:

The four human annotation tasks are intuitively reasonable, but their theoretical justification and annotation reliability remain insufficiently validated. In particular, what motivates the selection of the seven coarse-grained affective factors and the three fine-grained factors?

Authors’ Response:

We thank the reviewer for the valuable comment.

  • Theoretical justification for factor selection. We organize the seven coarse-grained affective factors into three levels according to their visual hierarchy and semantic roles in affective perception: | Level | Factors | Visual Information | |:-|:-|:-| | Low-level Visual Cues | Weather, Lighting, Color | Environmental conditions and appearance characteristics | | Mid-level Structural Cues | Composition, Perspective | Spatial layout, viewpoint, and visual organization | | High-level Semantic Cues | Elements, Theme | Scene entities, events, and semantic context |

    • Regarding the selection of the three fine-grained factors, Weather, Lighting, and Color are directly observable visual cues that can be characterized using consistent attributes across heterogeneous images.
    • In contrast, Composition, Perspective, Elements, and Theme are more context-dependent and lack stable criteria for fine-grained categorization across diverse scenes.
    • Therefore, we capture the former through detailed attribute labels and the latter through relative importance rankings.
  • Annotation Reliability. We further assess the inter-annotator agreement of ContiEmo through statistical analyses.

    • V/A ratings: we report single-rater intraclass correlation coefficient ICC(2,1) and five-rater averaged agreement ICC(2,5) to assess rating consistency.
    • Seven-factor importance rankings and three-teacher CoT preference rankings: we calculate Kendall's coefficient of concordance (W) to quantify ranking agreement.
    • Three fine-grained factors: we adopt Fleiss' kappa to evaluate categorical agreement.
    • Teacher preference: we additionally report top-1 preference agreement to quantify how frequently annotators identify the same teacher.

As shown in Table R3-1, five-rater aggregated V-A ratings demonstrate higher reliability than individual ratings, while the factor rankings and teacher preferences exhibit relatively high inter-annotator agreement. These analyses support the reliability of ContiEmo annotations.

We have added the theoretical justification and reliability results to Section 4.5 [Annotation Reliability and Justification of ContiEmo] of the revised manuscript.

Table R3-1: Inter-Annotator Reliability of ContiEmo Annotations.

Annotation TaskDimensionMetricScore
Affective RatingValenceICC(2,1)0.55
ICC(2,5)0.86
ArousalICC(2,1)0.48
ICC(2,5)0.83
Coarse-grained Factors7-factor RankingKendall's W0.61
Fine-grained FactorsWeatherFleiss' κ0.88
LightingFleiss' κ0.75
ColorFleiss' κ0.71
Teacher PreferenceCoT RankingKendall's W0.64
Top-1 PreferenceAgreement (%)72.0

Comment 2:

Further evaluation of the visual quality and semantic consistency of generated intermediate frames would strengthen the reliability of ContiEmo.

Authors’ Response:

Thank you for this suggestion. We conduct a human evaluation of the intermediate frames.

  • Specifically, we randomly sample 500 base-image–sequence pairs, with balanced coverage of the four editing directions. Each sequence is independently evaluated by three annotators on a 5-point scale in two aspects:

    • Semantic Consistency, assessing the preservation of the original image's main entities, scene identity, and spatial relationships.
    • Visual Quality, assessing image clarity, visual artifacts, structural distortions, and temporal coherence across intermediate frames.
  • We report the mean and std of the ratings, with inter-annotator agreement measured by ICC(2,1) and ICC(2,3). The results are presented in Table R3-2 and Table R3-3. The generated sequences achieve consistently high ratings in both semantic consistency and visual quality across all four editing directions, with good inter-annotator agreement.

Table R3-2: Human evaluation of intermediate-frame quality.

Editing DirectionSemantic ConsistencyVisual Quality
HVHA4.38 ± 0.524.20 ± 0.62
HVLA4.33 ± 0.554.12 ± 0.66
LVHA4.30 ± 0.574.08 ± 0.61
LVLA4.40 ± 0.504.17 ± 0.58

Table R3-3: Inter-annotator Agreement.

Evaluation DimensionICC(2,1)ICC(2,3)
Semantic Consistency0.770.91
Visual Quality0.720.89

We have added this evaluation to Appendix C.3 [Human Evaluation of the Intermediate Frames].

AuthorsOpenReview ↗

Response to Reviewer tqTf on More Evaluations and Comparisons. (2/2)

Comment 3:

Pairwise evaluation on homologous images would provide more direct evidence of the model's ability to capture fine-grained affective variations.

Authors’ Response:

Thank you for this comment. Our original ContiEmo test predictions already include homologous images. We conduct an additional pairwise analysis to assess the model's ability to capture fine-grained affective variations.

For each pair of images originating from the same base image, we compare the predicted and human-annotated score differences along Valence and Arousal independently. A prediction is considered correct if both differences have the same sign. We use group-balanced Pairwise Accuracy (PAcc) to ensure that each base-image group contributes equally to the overall evaluation.

  • Overall pairwise discrimination. Table R3-4 shows that EmoInsight achieves 77.3% Valence PAcc and 71.6% Arousal PAcc, surpassing all competing methods.

  • Sensitivity to affective difference magnitude. We further categorize image pairs into small (0 < |Δy| ≤ 0.5), moderate (0.5 < |Δy| ≤ 1.0), and large (|Δy| > 1.0) groups. As shown in Table R3-5, EmoInsight obtains better performance than the strongest baseline Qwen3-VL-8B across all three groups, demonstrating its ability to distinguish even subtle affective variations.

The pairwise results provide evidence that EmoInsight captures fine-grained affective variations within homologous images. We have incorporated the evaluation protocol and findings into Section 4.4.3 [Effectiveness of Dense Homologous Frames] of the revised manuscript.

Table R3-4: Overall Pairwise Ordering Accuracy.

MethodV-PAcc ↑A-PAcc ↑
StyleEDL60.654.8
EmoSet68.061.7
VA-CLIP62.957.4
MFRN66.462.3
Qwen3-VL-4B64.860.0
Qwen3-VL-8B68.763.5
EmoInsight77.371.6

Table R3-5: Ordering Accuracy under Different Affective Differences.

MethodV-Small ↑V-Moderate ↑V-Large ↑A-Small ↑A-Moderate ↑A-Large ↑
Strongest Baseline60.267.577.656.063.174.3
EmoInsight70.476.183.265.872.080.1

Comment 4:

Comparisons with top-ranked CoT selection and learnable adaptive weighting would better validate the effectiveness of multi-teacher distillation.

Authors’ Response:

Thank you for this comment. We have extended the multi-teacher distillation ablation with two comparisons:

  • Top-ranked CoT Selection, which retains only the highest-ranked teacher CoT for each image based on human preferences, equivalent to a one-hot teacher weighting.
  • Learnable Adaptive Weighting, which employs a two-layer MLP (Linear(d, 256)-GELU-Linear(256, 3)) to predict image-conditioned teacher weights from mean-pooled visual features. The MLP is trained using the human preference distributions with a KL-divergence loss, and is then used to generate teacher weights for student SFT.

As shown in Table R3-6, our Dynamic Soft Weighting outperforms both Top-ranked CoT Selection and Learnable Adaptive Weighting, demonstrating that retaining the full human preference distribution is more effective than hard teacher selection or learnable weighting in multi-teacher distillation.

Table R3-6: Comparisons with Top-ranked CoT Selection and Learnable Adaptive Weighting.

MethodV-Err ↓A-Err ↓
Uniform Weighting0.7110.349
Global Fixed Weighting0.7010.327
Top-ranked CoT Selection0.6280.321
Learnable Adaptive Weighting0.6230.316
Ours: Dynamic Soft Weighting0.6140.309

We have supplemented the additional multi-teacher distillation ablation in Section 4.4.1 [Effectiveness of Multi-Teacher Synergistic Affective Reasoning mechanism] of the revised manuscript.


We have conducted extensive evaluations and analyses to address all the concerns, and sincerely hope the reviewer can reconsider the overall score based on these revisions.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer tqTf,

We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.

Could you please kindly let us know if our rebuttal has addressed your concerns?

If you have any further questions regarding our work, we would be delighted to address them.

If our responses have addressed your concerns, we would greatly appreciate your reconsideration of your overall assessment.

Best regards,

Authors

Dear Reviewer tqTf,

As the rebuttal deadline is just over 24 hours away, we would greatly appreciate any feedback on our responses.

Thank you!

Reviewer_LMHUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ContiEmo, a VEA dataset of 105K images. It is built by editing 5K EmoSet images toward four V-A quadrants and sampling intermediate frames from first–last-frame generated videos. Each image is annotated by five annotators with V-A scores, factor importance rankings, fine-grained factor labels, and preference rankings over three teacher CoTs. On top of this dataset, the authors propose EmoInsight, a Qwen3-VL-4B model trained in two stages: multi-teacher CoT distillation weighted by human preferences, followed by factor-guided GRPO with decoupled reward normalization. Experiments on ContiEmo, EMOTIC, and FindingEmo report improvements in V-A estimation and factor recognition.

Critical strengths

S1. A dataset with an interesting construction and rich annotation. Generating homologous image groups through affective editing and video interpolation, then calibrating every image with human V-A ratings, is an interesting way to obtain samples that share semantics but vary in affect. Combined with factor and CoT preference annotations, this is the main potential contribution of the paper and the primary reason it merits consideration, although its validity remains to be established (W4–W7).

S2. Consistent gains with significance testing. EmoInsight improves over all baselines on all three datasets, and the gains are supported by paired bootstrap tests with Holm correction. Table 5 also attempts to separate the effect of homologous frames from that of data size. However, the strength of this evidence is limited by the baseline concerns in W8.

Critical weaknesses

Motivation

W1. Limited novelty of the problem. Using reasoning to improve perceptual assessment is a well-established direction in visual reasoning, and MLLM-based emotion reasoning, including RL-based approaches, already exists. These works are neither discussed nor compared, so introducing reasoning into VEA does not by itself constitute a novel problem.

Method

W2. Limited methodological novelty. Multi-teacher CoT distillation, the SFT→GRPO pipeline, thresholded score rewards, and per-reward advantage normalization are all established techniques. Moreover, the preference-based weights are bounded in [1/6, 1/2] by Eq. 1. Yet in Table 3, uniform weighting is worse than every single teacher while dynamic weighting beats all of them, which is hard to reconcile with such mild reweighting.

W3. The role of reasoning is unclear. The RL prompt forbids any explanation (Table 8) and limits the completion length to 256 tokens (Table 7). This contradicts the claim in Sec. 3.3 that the model generates a CoT at inference time. In addition, no ablation without CoT is provided, so the benefit of reasoning is not demonstrated.

Dataset

W4. Questionable validity of the emotion editing. In Fig. 8(a), the human-annotated endpoints fall mainly into two diagonal clusters, and HVLA is almost absent, so the four-quadrant editing is not realized. Furthermore, the edits mainly change lighting, color, and weather, which are exactly the factors used for supervision. This raises a concern of circularity.

W5. The homologous structure is not exploited. Homologous images are used as independent training samples, and no objective makes use of within-group relations. The value of homology rests solely on Table 5.

W6. Annotation reliability is not reported. Each annotator handles roughly 10K images with multiple annotation tasks per image. However, no inter-annotator agreement or quality control is reported, which is essential for a dataset contribution.

Experiments

W7. No evidence that ContiEmo benefits real-world data. EMOTIC and FindingEmo are trained and tested independently, with no cross-dataset transfer. The claimed “generalization” therefore does not reflect the value of ContiEmo.

W8. Questionable baselines. Several baselines show negative or near-zero Spearman correlations (e.g., StyleEDL A-SR −0.402), suggesting improper adaptation. In addition, the training of the Qwen3-VL baselines is unspecified, and neither MLLM-based emotion methods nor the teacher models themselves are compared.

Questions for authors

Q1. Dataset construction

  • What proportion of edited images are annotated within their target V-A quadrant, and why do the HVLA and LVHA edits largely fail?
  • What is the inter-annotator agreement for the V-A ratings, the factor rankings, and the CoT preferences?
  • In Table 5, are the "random images" intermediate frames or edited endpoints?

Q2. Training details

  • Does the final model actually output a CoT, given the RL prompt and the 256-token limit?
  • How is the multi-teacher SFT stage performed on EMOTIC and FindingEmo, which have no preference annotations?
  • How are the Qwen3-VL-4B/8B baselines trained?
7 comments
AuthorsOpenReview ↗

Response to Reviewer LMHU on More Method, Dataset and Training Details. (1/5)

Thank you very much for recognizing the novel construction of ContiEmo and the consistent gains of EmoInsight. In the following, we provide our point-by-point responses to your comments and describe the corresponding revisions in our paper.

Question 1-1 & W4:

What proportion of edited images are annotated within their target V-A quadrant, and why do the HVLA and LVHA edits largely fail?

Questionable validity of the emotion editing. In Fig. 8(a), the human-annotated endpoints fall mainly into two diagonal clusters, and HVLA is almost absent, so the four-quadrant editing is not realized. Furthermore, the edits mainly change lighting, color, and weather, which are exactly the factors used for supervision. This raises a concern of circularity.

Authors’ Response:

Thank you for this comment.

  • We calculate the target-quadrant proportion of affective editing in Table R2-1. An edited image is considered aligned with its target quadrant if its annotated V-A coordinates fall within the corresponding quadrant, using (V=3, A=3) as the boundary.
    • We emphasize that we do not artificially enforce a balanced distribution across the four quadrants. Instead, we preserve the actual affective perceptions reported by annotators.
    • The relatively sparse HVLA and LVHA region reflects the difficulty of eliciting positive yet suppressed and negative yet activated affective perceptions through image editing while preserving the main semantics.

Table R2-1: Target-Quadrant Proportion of Affective Editing.

Editing DirectionCharacteristicsProportion (%)
HVHAPositive and excited33.8
HVLAPositive yet suppressed9.6
LVLANegative and subdued36.1
LVHANegative yet activated20.5
  • Clarification of No Circularity between Editing and Factor Supervision.
    • Our emotion editing is conditioned on target V-A directions, without explicitly prescribing changes in weather, lighting intensity, or color richness to achieve these targets.
    • Our supervision extends beyond these three fine-grained factors to human rankings of seven affective factors (weather, light, color, composition, elements, perspective, and theme).

We have added the target-quadrant proportion analysis and no circularity clarification to Appendix C.1 [Dataset V-A Distribution] of the revised manuscript.


Question 1-2 & W6:

What is the inter-annotator agreement for the V-A ratings, the factor rankings, and the CoT preferences?

Annotation reliability is not reported. Each annotator handles roughly 10K images with multiple annotation tasks per image. However, no inter-annotator agreement or quality control is reported, which is essential for a dataset contribution.

Authors’ Response:

Thank you for this comment.

To establish the reliability of ContiEmo annotations, we have conducted statistical analyses of inter-annotator agreement across four annotation tasks:

  • V/A ratings: we use the intraclass correlation coefficient (ICC) to measure rating consistency. We report both single-rater ICC(2,1) and five-rater averaged agreement ICC(2,5).
  • Seven-factor importance rankings and three-teacher CoT preference rankings: we calculate Kendall's coefficient of concordance (W) to measure ranking agreement.
  • Three fine-grained factors: we report Fleiss' kappa to measure categorical agreement.
  • Teacher preference: we additionally report top-1 preference agreement to quantify how frequently annotators identify the same teacher.

As shown in Tables R2-2, the five-rater aggregated V-A labels demonstrate higher reliability than individual ratings, while the factor rankings and teacher preferences have relatively high inter-annotator agreement.

Table R2-2: Inter-Annotator Reliability of ContiEmo Annotations.

Annotation TaskDimensionMetricScore
Affective RatingValenceICC(2,1)0.55
ICC(2,5)0.86
ArousalICC(2,1)0.48
ICC(2,5)0.83
Coarse-grained Factors7-factor RankingKendall's W0.61
Fine-grained FactorsWeatherFleiss' κ0.88
LightingFleiss' κ0.75
ColorFleiss' κ0.71
Teacher PreferenceCoT RankingKendall's W0.64
Top-1 PreferenceAgreement (%)72.0

We have added the annotation reliability results to Section 4.5 [Annotation Reliability and Justification of ContiEmo] of the revised manuscript.


Question 1-3:

In Table 5, are the "random images" intermediate frames or edited endpoints?

Authors’ Response:

Thank you for this question.

The random images are intermediate frames randomly sampled from different base-image groups, with their V-A distribution and sample count matched to those of the homologous intermediate frames.

We have clarified the meaning of "random images" in Section 4.4.3 [Effectiveness of Dense Homologous Frames] of the revised manuscript to avoid confusion.

AuthorsOpenReview ↗

Response to Reviewer LMHU on More Method, Dataset and Training Details. (2/5)

Question 2-1 & W3:

Does the final model actually output a CoT, given the RL prompt and the 256-token limit?

The role of reasoning is unclear. The RL prompt forbids any explanation (Table 8) and limits the completion length to 256 tokens (Table 7). This contradicts the claim in Sec. 3.3 that the model generates a CoT at inference time. In addition, no ablation without CoT is provided, so the benefit of reasoning is not demonstrated.

Authors’ Response:

Thank you for this comment.

  • The final model generates CoT during inference, with the maximum generation length increased to 1024 tokens.
  • The final inference prompt follows the RL prompt, with only the addition of a thinking process field, while all other instructions remain unchanged:
You are an expert in visual emotion assessment. Analyze the provided image according to the following instructions:

1. Weather Condition: Identify the weather using one of the following categories: Sunny, Cloudy, Overcast, Rainy, Foggy, Snowy, Thunderstorm, or Indoor.

2. Lighting Intensity: Evaluate the lighting intensity using one of the following levels: Very Strong, Strong, Moderate, Weak, or Very Weak.

3. Color Richness: Evaluate the color richness using one of the following levels: Very Rich, Rich, Moderate, Monotonous, or Very Monotonous.

4. Factor Ranking: Rank the following seven factors according to their importance in determining the image's Valence and Arousal, from most important to least important: Weather, Light, Color, Composition, Elements, Perspective, and Theme. Each factor must appear exactly once.

5. Emotion Rating: Rate Valence and Arousal on a continuous scale from 1.0 to 5.0. For Valence, 1.0 represents very negative affect and 5.0 represents very positive affect. For Arousal, 1.0 represents very calm affect and 5.0 represents very active affect. Report both values as floating-point numbers rounded to one decimal place.

Output Requirement: Return the result strictly and exclusively as a single JSON object enclosed by <answer> and </answer> tags using the following structure:

<answer>
{
  "thinking process": "<your reasoning process>",
  "weather": "<weather category>",
  "light": "<lighting intensity>",
  "color": "<color richness>",
  "rank": ["factor1", "factor2", "factor3", "factor4", "factor5", "factor6", "factor7"],
  "valence": <rating>,
  "arousal": <rating>
}
</answer>

Do not include any conversational text or Markdown code blocks inside or outside the answer tags.
  • CoT plays different roles across the three stages of EmoInsight:

    • SFT: Explicit CoT supervision transfers multi-cue affective reasoning knowledge from teachers to the student.
    • RL:No explicit CoT is generated, allowing the model to concentrate on factor prediction and V-A assessment.
    • Final Inference: CoT is reintroduced to provide interpretable affective reasoning alongside structured predictions.
  • To examine the contribution of CoT distillation, we introduce a w/o CoT Distillation baseline by directly applying FED-GRPO to the original Qwen3-VL-4B model. We keep the RL dataset, prompts, reward functions, and training configuration identical to those of the full model. Both models are evaluated using the same inference prompt. As shown in Table R2-3, the full EmoInsight outperforms the w/o CoT baseline, demonstrating the benefits of reasoning in improving factor prediction and affective assessment.

Table R2-3: Ablation without CoT.

MethodV-Err ↓A-Err ↓V-SR ↑A-SR ↑
w/o CoT baseline0.4220.2850.7030.418
EmoInsight0.3710.2460.7670.500

We have clarified the CoT prompt during final inference in Appendix B [Prompt Templates and Output Schema], and added an ablation without CoT in Section 4.2.2 [Effectiveness of Coarse-to-Fine Factor-Guided Reinforcement strategy].


Question 2-2:

How is the multi-teacher SFT stage performed on EMOTIC and FindingEmo, which have no preference annotations?

Authors’ Response:

Thank you for this comment.

  • Multi-teacher SFT is performed only on ContiEmo, which provides teacher-generated CoTs and human preference annotations for dynamic weighting. It is not performed on EMOTIC or FindingEmo, which lack these annotations.

  • This is what we intended by stating that "the training objectives are adapted to the annotations in each dataset." We have explicitly clarified the dataset-specific application of multi-teacher SFT in Section 4.1 [Baselines and Datasets] of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer LMHU on More Method, Dataset and Training Details. (3/5)

Question 2-3 & W8:

How are the Qwen3-VL-4B/8B baselines trained?

Questionable baselines. Several baselines show negative or near-zero Spearman correlations (e.g., StyleEDL A-SR −0.402), suggesting improper adaptation. In addition, the training of the Qwen3-VL baselines is unspecified, and neither MLLM-based emotion methods nor the teacher models themselves are compared.

Authors’ Response:

Thank you for this comment.

We provide additional implementation details and teacher comparisons to clarify our evaluation protocol.

  • Baseline and negative SR values.
    • We strictly follow the official implementations and recommended hyperparameter settings of the specialized VEA baselines, with only dataset-specific adaptations to the V–A scales and factor annotations.
    • Regarding the negative A-SR values of StyleEDL and VA-CLIP, Spearman correlation measures rank-order consistency rather than absolute prediction accuracy. This can occur particularly when homologous images exhibit small affective differences, as even modest prediction errors can reverse their relative ordering.
  • Table R2-4: Training Details of Qwen3-VL-4B/8B Baselines. Both models are trained and evaluated using the same dataset splits as EmoInsight. | Configuration | Qwen3-VL-4B/8B | |:-|:-| | Initialization | Official pretrained checkpoint | | Fine-tuning Strategy | LoRA-based SFT | | LoRA Rank / Alpha | 64 / 128 | | Optimizer | AdamW | | Learning Rate | 1e-4 | | Batch Size | 16 | | Training Epochs | 3 | | Prompt | Same as the prompt of Coarse-to-Fine Factor-Guided Reinforcement |
  • Comparison with teacher models.
    • To provide MLLM references for continuous V-A assessment, we evaluate Qwen-VL-Max, GPT-5.2, and Gemini 3 Pro teacher models on ContiEmo using the same prompt as Coarse-to-Fine Factor-Guided Reinforcement. We report V-Err, A-Err, V-SR, A-SR, and the factor accuracies (W/L/C-ACC).
    • As shown in Table R2-5, the three teachers exhibit complementary strengths across affective dimensions and visual factors. Moreover, EmoInsight achieves better performance than all teachers.

Table R2-5: Comparison with MLLM-based Teacher Models.

MethodV-Err ↓A-Err ↓V-SR ↑A-SR ↑W-ACC ↑L-ACC ↑C-ACC ↑
Qwen-VL-Max0.6390.3250.7380.4610.8340.3880.284
GPT-5.20.6310.3020.7260.3570.8750.5130.313
Gemini3 Pro0.6070.3640.7120.2080.6340.3880.400
EmoInsight0.3710.2460.7670.5000.9340.6060.625

We have added the corresponding experimental results and implementation details to Appendix A.4 [Baseline Details] of the revised manuscript.


Other Weaknesses:

W1. Limited novelty of the problem. Using reasoning to improve perceptual assessment is a well-established direction in visual reasoning, and MLLM-based emotion reasoning, including RL-based approaches, already exists. These works are neither discussed nor compared, so introducing reasoning into VEA does not by itself constitute a novel problem.

Authors’ Response:

Our motivation is not merely to introduce reasoning into visual emotion assessment. Rather, we aim to achieve fine-grained continuous affective assessment by leveraging homologous images with hierarchical human preferences. Specifically, EmoInsight differs from existing reasoning-enhanced perceptual assessment [1][2] and MLLM-based emotion reasoning [3] in three aspects:

AspectReasoning-enhanced Perceptual AssessmentMLLM-based Emotion ReasoningEmoInsight (Ours)
Primary ObjectiveImprove visual perception through reasoningUnderstand and explain emotional contentFine-grained continuous V-A assessment
Supervision StructureTask-specific perceptual annotations and visual evidence supervisionEmotion labels and rationale-based explanationsJoint annotations of homologous affective variations, human CoT preferences, and hierarchical factors
Reasoning RoleReason over task-relevant visual evidenceIdentify emotional states and their visual triggersTransfer human-preferred reasoning into factor-grounded V-A optimization

We have compared the relevant works with our method and clarified our novelty in Section 2 [Related Work] of the revised manuscript.


Reference:

[1] Bigverdi M, et al. Perception tokens enhance visual reasoning in multimodal language models[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 3836-3845.

[2] Xiao T, et al. Perception-r1: Advancing multimodal reasoning capabilities of mllms via visual perception reward[C]//International Conference on Learning Representations. 2026, 2026: 26868-26898.

[3] Cheng Z, et al. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning[J]. Advances in Neural Information Processing Systems, 2024, 37: 110805-110853.

Reviewer_brtyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ContiEmo, a visual emotion assessment dataset containing 105,000 images derived from 5,000 source images. Image editing and video generation produce related images with different affective appearances. Human annotations include valence, arousal, visual factors, and preferences over teacher-generated explanations. The proposed EmoInsight model first learns from multiple teachers using preference-weighted supervision, then undergoes reinforcement learning with emotion and factor rewards. Each image is assessed independently. Experiments on ContiEmo, EMOTIC, and FindingEmo report improvements in affect prediction and factor recognition, with ablations examining teacher weighting, reward design, and intermediate-frame supervision.

Critical strengths
  1. ContiEmo provides a useful way to study visual-affective variation while preserving scene correspondence. The dataset combines related images with multiple forms of human supervision. The experimental protocol also handles their dependence appropriately: all derivatives of a source image remain in the same split, and statistical testing resamples source-image groups. These choices strengthen confidence that the results are not driven by related frames appearing across training and test sets.

  2. The intermediate-frame ablation includes a meaningful control. Table 5 compares intermediate frames against an equal number of random images matched to their valence–arousal distribution. Intermediate frames reduce valence error from 0.428 to 0.371 and increase arousal rank correlation from 0.457 to 0.500. This provides evidence that the benefit goes beyond adding more training images. The teacher-weighting and reward ablations offer further support for the training design, while Table 1 reports improvements under separate training on two public datasets.

Critical weaknesses
  1. The reliability of the fine-grained human annotations is not established. Each image receives five ratings, and Appendix C explains aggregation and tie handling. However, the paper does not report rating dispersion, inter-annotator reliability, preference-ranking agreement, or annotation-quality checks. These statistics are particularly important here because the dataset is intended to capture small affective differences between closely related images. Averaging ratings does not show whether those differences are reproducible or comparable to variation between annotators. Subjective disagreement is expected; the missing evidence is how much uncertainty remains in the aggregated labels and how that uncertainty affects the benchmark.

  2. The evaluation does not directly test discrimination within homologous image groups. The reported errors and rank correlations pool images across scenes. They can improve through better prediction of broad differences between scenes or affective quadrants without establishing sensitivity to smaller changes within a group. Table 5 supports the value of the training data, but a within-group ordering or affective-difference measure would test the paper’s defining capability more directly. This distinction also matters when interpreting the public-dataset results: those models are trained separately on each dataset, so the results demonstrate the framework’s usefulness after adaptation rather than transfer from ContiEmo to unseen real-world data. The absence of direct within-group evaluation is a main reason for my current recommendation.

Questions for authors
  1. Can the authors provide the existing annotation-reliability statistics, including valence–arousal dispersion and agreement on teacher preferences? In particular, how do affective differences between nearby variants compare with variation between annotators?

  2. Using the existing test predictions, can the authors report a within-group metric, such as ordering accuracy among variants from the same source image, for EmoInsight and the strongest baseline? Results on variants with small human-rated differences would help determine whether the aggregate improvements reflect the claimed fine-grained sensitivity.

6 comments
AuthorsOpenReview ↗

Response to Reviewer brty on Annotation Reliability and Within-Group Fine-Grained Sensitivity. (1/2)

Thank you very much for your positive assessment of our useful dataset design and comprehensive experimental validation.

Your constructive comments and suggestions are exceedingly helpful to improve our paper. We have carefully incorporated them in the revised paper. In the following, your comments are first stated and then followed by our point-by-point responses.

Comment 1:

Can the authors provide the existing annotation-reliability statistics, including valence–arousal dispersion and agreement on teacher preferences? In particular, how do affective differences between nearby variants compare with variation between annotators?

Authors’ Response:

Thank you for raising this comment. We have conducted additional statistical analyses of the original annotations.

  • Inter-annotator agreement across four annotation tasks.
    • For the V/A ratings, we use intraclass correlation coefficient (ICC) to measure the consistency of ratings. We report both single-rater ICC(2,1) and five-rater averaged agreement ICC(2,5).
    • For the seven-factor importance rankings and the three-teacher CoT preference rankings, we calculate Kendall's coefficient of concordance (W) to measure ranking agreement.
    • For the three fine-grained factors, we report Fleiss' kappa to measure categorical agreement.
    • For the teacher preference, we report top-1 preference agreement to quantify how frequently annotators identify the same teacher.
  • Relationship between within-group affective differences and inter-annotator variation. We further investigate whether affective differences between homologous images are distinguishable from annotation variability. Specifically, we report the mean and median of three statistics:
    • Per-image inter-annotator standard deviation, measuring rating dispersion among the five annotators;
    • Pairwise affective difference, measuring the absolute V-A score difference between nearby homologous images;
    • Difference-to-dispersion ratio, measuring the pairwise affective difference relative to the pooled inter-annotator dispersion of the corresponding images.

The results are presented in Tables R1-1 and R1-2. Overall, the five-rater aggregated V-A labels exhibit higher reliability than individual ratings, while the factor rankings and teacher preferences show relatively high inter-annotator agreement. The nearby affective differences exceed the annotation dispersion.

These analyses provide a more explicit characterization of the annotations reliability. We have added the reliability analyses to Section 4.5 [Annotation Reliability and Justification of ContiEmo] and Appendix C.4 [Further Annotation Reliability] of the revised manuscript.

Table R1-1: Inter-Annotator Reliability of ContiEmo Annotations.

Annotation TaskDimensionMetricScore
Affective RatingValenceICC(2,1)0.55
ICC(2,5)0.86
ArousalICC(2,1)0.48
ICC(2,5)0.83
Coarse-grained Factors7-factor RankingKendall's W0.61
Fine-grained FactorsWeatherFleiss' κ0.88
LightingFleiss' κ0.75
ColorFleiss' κ0.71
Teacher PreferenceCoT RankingKendall's W0.64
Top-1 PreferenceAgreement (%)72.0

Table R1-2: Homologous Images Differences and Annotation Variability.

DimensionStatisticMeanMedian
ValenceInter-annotator Std0.230.20
Nearby Affective Difference0.390.33
Difference/Dispersion Ratio1.721.58
ArousalInter-annotator Std0.250.24
Nearby Affective Difference0.440.39
Difference/Dispersion Ratio1.741.63
AuthorsOpenReview ↗

Response to Reviewer brty on Annotation Reliability and Within-Group Fine-Grained Sensitivity. (2/2)

Comment 2:

Using the existing test predictions, can the authors report a within-group metric, such as ordering accuracy among variants from the same source image, for EmoInsight and the strongest baseline? Results on variants with small human-rated differences would help determine whether the aggregate improvements reflect the claimed fine-grained sensitivity.

Authors’ Response:

Thank you for this suggestion. We have conducted a within-group pairwise evaluation.

Specifically, we construct image pairs from the same base-image group and evaluate their relative ordering separately along the Valence and Arousal dimensions. A pair is considered correctly ordered when the sign of the predicted score difference agrees with that of the human-annotated score difference. We report group-balanced Pairwise Accuracy (PAcc), ensuring that each base-image group contributes equally to the overall evaluation.

  • Overall within-group ordering accuracy. As shown in Table R1-3, EmoInsight achieves a Valence PAcc of 77.3% and an Arousal PAcc of 71.6%, outperforming all baselines.
  • Ordering accuracy under different affective differences. To examine sensitivity to subtle affective changes, we further stratify image pairs according to their absolute V-A differences into small (0 < |Δy| ≤ 0.5), moderate (0.5 < |Δy| ≤ 1.0), and large (|Δy| > 1.0) groups. As reported in Table R1-4, EmoInsight consistently outperforms the strongest baseline (Qwen3-VL-8B) across all three difference levels.

These results provide direct evidence that EmoInsight can distinguish fine-grained affective variations among homologous images. We have added the evaluation protocol, results, and corresponding discussion to Section 4.4.3 [Effectiveness of Dense Homologous Frames] of the revised manuscript.

Table R1-3: Overall Within-Group Ordering Accuracy.

MethodV-PAcc ↑A-PAcc ↑
StyleEDL60.654.8
EmoSet68.061.7
VA-CLIP62.957.4
MFRN66.462.3
Qwen3-VL-4B64.860.0
Qwen3-VL-8B68.763.5
EmoInsight77.371.6

Table R1-4: Ordering Accuracy under Different Affective Differences.

MethodV-Small ↑V-Moderate ↑V-Large ↑A-Small ↑A-Moderate ↑A-Large ↑
Strongest Baseline60.267.577.656.063.174.3
EmoInsight70.476.183.265.872.080.1

We have carefully addressed all concerns with supplementary experiments and analyses, and sincerely hope the reviewer can reconsider the overall score in light of these revisions.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer brty,

We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.

Could you please kindly let us know if our rebuttal has addressed your concerns?

If you have any further questions regarding our work, we would be delighted to address them.

If our responses have addressed your concerns, we would greatly appreciate your reconsideration of your overall assessment.

Best regards,

Authors

Comments on the paper

1 comment

Dear Reviewers,

We’re now past the halfway point of the ICLR discussion period. Please take a moment to read the authors’ responses carefully and let them know which of your concerns have been addressed and which questions remain open. If anything needs clarification or would benefit from further discussion, please raise it and engage with the authors and fellow reviewers while there’s still time.

Thank you for your time, your thoughtful reviews, and your service to the ICLR community.

Best Regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.