acceptodds
Withdrawn from ICLR 2027

Same Object, Different Emotions: Object-Centric Emotional Content Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-to-image models can generate semantically coherent images, yet often struggle to make a specified object convey a desired emotion. Existing methods typically rely on scene-level cues or distort the object to strengthen emotional expression, compromising object identity and realism. We address object-based emotion-controllable image generation, where both the target object and the overall image should express the intended emotion while remaining visually plausible. To support this task, we construct Obj-EmoSet, which augments affective images with object categories, basic content descriptions, and emotion-aware descriptions. We then propose an Emotion-Aware Visual Planner that translates abstract emotions into object-compatible states, actions, and scene conditions. A Visual Realism Refiner improves composition, spatial relationships, and textures while retaining emotional cues, and an Emotion Calibration Adapter aligns emotional semantics with visual features. Together, these components enable object-level emotional expression while preserving semantic consistency and visual realism. Extensive qualitative and quantitative experiments demonstrate strong performance in object-level emotion control, semantic consistency, and image realism, advancing emotion-aware generation from coarse global modulation toward fine-grained object-centric control.

Primary area:
generative models
Keywords:
Object-based emotion-controllable generation, Emotion-controllable image generation, Subject consistency

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_svQSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies object-centric emotional image generation. It constructs Obj-EmoSet, which augments affective images with object categories and paired basic and emotion-aware descriptions, and proposes a framework comprising an Emotion-Aware Visual Planner, a Visual Realism Refiner, and an Emotion Calibration Adapter to translate target emotions into object-related visual conditions. Validation includes image-level and segmented object-level comparisons, module ablations, and a user study, reporting improved emotional alignment and favorable human preferences over the evaluated baselines.

Critical strengths

The problem has practical significance. Emotional image generators may rely on background atmosphere or implausible object deformation to convey emotions. Achieving emotional control while preserving object semantics and visual plausibility is a valuable and challenging research direction.

Critical weaknesses

W1: The ablation results do not support the necessity of the complete three-module framework. In Table 4, the Planner-only configuration matches the full model in Emo-A and performs better in EC-A, semantic consistency, and generation quality. Moreover, adding Cali worsens MLLM-P and Style. These results do not support the claim that the three modules jointly achieve a better balance between emotional expression and realism. The authors should verify the results, report variability across runs, and clarify the consistent benefits contributed by the Refiner and Adapter.

W2: The validity of object-level emotional supervision remains unclear. Obj-EmoSet augments image-level affective data with object categories and textual descriptions. However, an image-level emotion label cannot be attributed to every object in the image, as the emotion we perceive can also arise from people, background, or events. The data construction procedure does not sufficiently explain how the association between the target object and emotion is verified, and the authors need to demonstrate that the dataset provides valid object-level emotional supervision.

Questions for authors

See Weaknesses.

Reviewer_VWV9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an object-centric emotion-controllable image generation method that enables both the specified object itself and the overall image to express a target emotion while preserving the object's identity and realism. The four main contributions are: (1) Obj-EmoSet, a dataset built upon EmoSet that provides pairs of "basic content descriptions" and "emotion-aware content descriptions"; (2) an Emotion-Aware Visual Planner that autoregressively generates a 4-slot structured plan from the target emotion and basic description; (3) a Visual Realism Refiner that rewrites the plan into a natural language prompt tailored for diffusion models; and (4) an Emotion Calibration Adapter that trains only the LoRA parameters using a flow-matching loss while keeping the FLUX.2 base model frozen.

Critical strengths

The identification of the "affection gap" and the failure modes of prior methods illustrated in Figure 1 are highly compelling. Furthermore, the task of emotion-controllable generation while preserving object identity addresses a clear practical demand.

Critical weaknesses
  1. In Table 4, the Plan module alone performs comparably to the full model in Emo-A and exceeds it in the other seven metrics. While Cali is designed to improve the balance between realism and emotional expression, adding Cali to Plan+Vis seems to degrade the realism metrics (MLLM-P and Style). This appears to contradict the statement in the text that “Cali provides a better balance.” Additionally, since Vis alone scores lower than the baseline in Emo-A, it is somewhat challenging to confirm the empirical benefits of Contributions (2) and (3) based on the current ablation data.

  2. There are noticeable numerical differences for the full model and the baseline (FLUX.2) across all metrics when comparing Table 1 and Table 4. These gaps seem larger than typical random variance. If different evaluation sets or conditions were used, it would be very helpful to include a brief explanation in the text. Without this clarification, it is difficult to determine which results to reference; for example, the full model's Style score in Table 4 is lower than the FLUX.2 score in Table 1, which slightly weakens the claim about optimal Style performance.

  3. EmoCtrl is a closely related work that shares an almost identical task and data design (content, emotion, emotion-aware prompt). Although the text mentions that EmoCtrl was included in the user study, it does not appear in Table 2, nor is it present in Tables 1 or 3. Furthermore, since the proposed method essentially expands "(emotion, object)" into detailed prompts for a LoRA-adapted FLUX.2, it would help to clarify the exact prompts given to the baseline models. If the baselines received only short prompts (e.g., "emotion word + object name"), the performance gap in Table 1 might largely stem from prompt enrichment. To more clearly demonstrate the novelty and the necessity of learning the Plan/Vis modules, it would be highly beneficial to include comparisons with: (a) EmoCtrl, (b) FLUX.2 with zero-shot prompt expansion via a general LLM, and (c) FLUX.2 with LoRA trained solely on emotion-aware descriptions.

Questions for authors

Q1. In Table 4, the Plan module alone performs comparably to, or even better than, the full model. How do you interpret this result, and why retain the Vis and Cali modules?

Q2. Why do the metric scores for "Ours" and "FLUX.2" in Table 1 differ completely from those of "Plan+Vis+Cali" and the "Baseline" in Table 4? Could you clarify what caused this discrepancy (e.g., different evaluation sets, evaluation conditions, or random seeds)?

Q3. Could you explain the rationale for excluding EmoCtrl from Tables 1–3? Specifically, doesn't its absence in Table 2 contradict the main text's claim that it was included in the user study?

Q4. What exact text prompts were provided to the baseline methods during the experiments? Additionally, are there any comparison results available for FLUX.2 utilizing zero-shot prompt expansion generated by a general-purpose LLM?

Q5. Which foundational language models are the Plan and Vis modules based on? Furthermore, how do you isolate the specific contributions of the Plan module from the inherent prior knowledge of these large LLMs?

Reviewer_RucxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The proposed approach is promising, and the evaluation on object regions is useful. However, it is still not clear whether the full model is better. A fair comparison with detailed prompts is also needed. I think these comparisons and more consistent reporting would make the main claims more convincing.

Critical strengths

Good problem. Distinguishing object-level emotional expression from global scene atmosphere is a useful research direction.

Structured data and planning. The paired Base Content / Emotion Content descriptions and structured visual plans provide a reasonable way to connect abstract emotions with observable visual attributes.

Critical weaknesses

The benefit of the full model is not clear. Plan-only gives Emo-A/EC-A of 53.10/50.78, while the full model gives 53.10/50.47. Adding Cali to Plan+Vis also lowers MLLM-P from 0.7785 to 0.7432 and worsens Style from 22.50 to 27.11. These results do not clearly support the need for the full pipeline, especially Cali. (Table 4)

The reported results are not fully consistent. Table 1 gives 53.61/49.50 for the full model’s Emo-A/EC-A, but Table 4 gives 53.10/50.47, without clearly explaining the setting difference. Also, the appendix reports 6,784 images for each of 11 methods, which gives 74,624, not the stated 81,408. These points need correction or explanation so readers can understand the evaluation clearly. (Tables 1 and 4; Appendix A.2)

Questions for authors

Thanks for your efforts in this paper. You are working on a valuable problem. But I still have two critical questions:

What is the main benefit of Cali over Plan-only? In Table 4, the full model does not improve Emo-A or EC-A over Plan-only. Adding Cali to Plan+Vis also makes the realism scores worse. Could you explain which results support the need for Cali?

Could you clarify the evaluation settings and image count? The full-model results differ between Tables 1 and 4. Were different prompts, seeds, or settings used? Also, Appendix A.2 reports 6,784 images per method for 11 methods, which gives 74,624 rather than 81,408. Which total is correct?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.