acceptodds
Withdrawn from ICLR 2027

Diverse Narratives, Preserved Content: Controllable Affective Image Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Controllable affective image generation aims to generate images from content descriptions and target emotions. A given content description and emotion target can support multiple visual narratives, but such diversity should arise from variations in actions, interactions, and situations rather than changes to protected content. However, existing methods struggle to distinguish content that must be preserved from narrative elements that can vary. To address this challenge, we propose a content–narrative decomposition framework that preserves specified entities, attributes, and required relations while enabling diverse narrative expressions. Specifically, scene-graph supervision and cross-reconstruction encourage composable content and narrative representations. Conditional flow matching models a distribution of latent narrative prompts, which, together with protected content and target valence–arousal values, guide an emotion injection module to update narrative features. These features are then recombined with the content representation to condition a frozen SDXL generator. To support one-to-many narrative learning, we augment the existing training data with diverse emotional descriptions for each content–emotion pair. Experiments demonstrate that our method reduces content violations and improves constraint-satisfying narrative diversity while maintaining emotion controllability. Ablation studies confirm the complementary roles of content–narrative decomposition and flow-based narrative modeling.

Primary area:
generative models
Keywords:
Controllable affective image generation, Content–narrative decomposition, Conditional flow matching, Emotion control, Narrative diversity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NZUwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies controllable affective image generation under content constraints. It combines graph-supervised content–narrative decomposition with conditional flow matching to generate diverse affective narratives while preserving specified entities, attributes, and relations. The authors construct multi-narrative training data and generate images using a frozen SDXL model. Experiments show improved content preservation and valid narrative diversity while maintaining affective alignment.

Critical strengths

Joint evaluation is more appropriate than evaluating affect or diversity alone. CSR, JVR, and VNC respectively measure content constraint satisfaction, joint content–affect validity, and the number of distinct narratives among valid outputs. This avoids counting constraint-violating outputs as diversity gains and better reflects practical creative needs than relying solely on LPIPS or perceptual differences.

Critical weaknesses

W1: The contributions of model design and data augmentation are not sufficiently separated. The ablations primarily remove graph supervision, cross-reconstruction, or flow matching within the proposed framework, without evaluating baselines trained on the same augmented multi-narrative data. It therefore remains unclear how much of the improvement comes from the architecture rather than the additional supervision. Retraining the main baselines on identical training data would help validate the independent contribution of the proposed design.

W2: Continuous affective control lacks direct validation. The experiments primarily evaluate a discrete V/A grid, demonstrating alignment at multiple target points but not smooth, monotonic responses to continuously varying targets. I suggest that the authors vary V/A targets at finer intervals while keeping the content and noise fixed to assess the emotional monotonicity along different directions.

Questions for authors

See Weaknesses.

Reviewer_nss6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies controllable affective image generation when a fixed content request and a target valence-arousal (V/A) pair may admit multiple visually distinct situations. It distinguishes protected content—specified entities, attributes, quantities, and required relations, from visual narrative, actions, interactions, and contextual details that may vary without violating the request. The proposed method learns a content-narrative decomposition from paired basic and emotional descriptions. Protected-content scene graphs supervise the content representation, while self-reconstruction, cross-reconstruction, and content consistency encourage the content and narrative representations to be composable. A conditional flow-matching model then samples latent narrative prompts conditioned on content and V/A; an affect-conditioned narrative transformer and reconstructor map these prompts into the conditioning space of a frozen SDXL model. To create one-to-many supervision, the authors augment 39,843 source description pairs from OASIS, FindingEmo, and EMOTIC with protected-content graphs and three reviewed narrative descriptions per content-affect condition, yielding 119,529 narratives. Evaluation uses 150 held-out descriptions over a 5-by-5 V/A grid, four generated images per condition, and automated measures of affect, text-image alignment, image quality, protected-content satisfaction, joint validity, and valid narrative diversity. The reported results show substantially higher content satisfaction and valid narrative count than EmotiCrafter and GPT-5.5 + SDXL, and the ablations attribute content preservation to graph supervision/cross-reconstruction and diversity to flow-based prompt sampling.

Critical strengths
  1. A useful and operational problem formulation. The paper identifies an important failure mode in affective generation: apparent diversity can be obtained by changing requested content, while strict preservation can collapse the range of situations. Defining protected entities/attributes/relations separately from variable narratives turns this trade-off into a measurable task. JVR and VNC are consequential because they count diversity only among outputs that satisfy both content and affect constraints.

  2. A coherent model design with targeted ablations. Graph supervision and cross-reconstruction are well matched to content preservation, while conditional flow matching addresses one-to-many narrative generation independently of image noise. The ablation pattern is internally consistent: removing graph supervision or cross-reconstruction lowers CSR/JVR, whereas replacing flow matching with a deterministic predictor largely preserves validity but sharply reduces VNC. This provides useful evidence that the components serve different intended roles.

Critical weaknesses
  1. The experimental comparison is not sufficient to establish the motivation or the necessity of the proposed method. The paper’s related-work section discusses affective generation (including CoEmoGen, EmoCtrl, and feedback/RL approaches), instruction-based affective editing, structured control, and conditional diversity. It distinguishes editing by noting that the proposed setting has no source image. This is a useful conceptual distinction, but it does not answer the empirical question of whether current high-capacity general image models already provide strong content preservation, prompt following, and diverse situational generation. The main experiments use SDXL Base 1.0 and affect-control variants inherited from EmotiCrafter. “GPT-5.5 + SDXL” is not a comparison with a modern closed image generator: GPT-5.5 only rewrites the text and SDXL still generates the image. The paper does not test recent unified/open models such as FLUX.1 Kontext, Qwen-Image, OmniGen2, or Step1X-Edit, nor strong proprietary generation/editing systems such as GPT Image, Gemini image models, or Seedream. Although editing systems do not exactly match the no-source-image setting, they enable a natural two-stage alternative—generate a content-faithful seed image, then modify affect/narrative while preserving specified elements. Likewise, an LLM/VLM agent can generate several constraint-preserving narrative prompts, call a strong image model, and rerank under a fixed budget. Without such baselines, it is unclear whether the task remains unsolved in practice or whether the proposed architecture improves over modern prompting, editing, and selection pipelines. This directly weakens the paper’s motivation, practical significance, and evidence of effectiveness.

  2. The current evidence also confounds architecture, additional supervision, and automated evaluation. The proposed model receives a large, model-generated three-narrative augmentation, while it is unclear whether any learned baseline is retrained with comparable supervision. A matched-data baseline such as sampling an accepted narrative description and prompting the same generator, or retraining EmotiCrafter on the augmented data is needed to separate architectural gains from data/prompting gains. Moreover, CSR, JVR, and VNC are determined by three runs of the same stated AI visual-assessor model, without a dated snapshot, human image-level validation, agreement statistics, or sensitivity analysis; affect is measured with FindingEmo predictions rather than human responses. Because content relations, narrative identity, and affect can be ambiguous, conservative voting does not establish evaluator accuracy. The paper additionally omits the exact Qwen model, important training/compute details, candidate acceptance and revision rates, and confidence intervals. These issues substantially limit confidence in both the measured gains and their reproducibility.

Questions for authors
  1. Modern foundation-model baselines: Under exactly the same prompts, V/A targets, output budget, and evaluation protocol, how does the method compare with representative current open and proprietary image models? In particular, can the authors evaluate (i) direct text-to-image generation with strong modern models, (ii) LLM/VLM-generated constraint-preserving narrative prompts followed by those models, and (iii) where applicable, a generate-then-edit pipeline that changes affect/narrative while explicitly preserving the requested entities, attributes, and relations? If these systems already perform strongly, what measurable capability remains unique to the proposed framework?

  2. Matched supervision and recent affective baselines: Can the authors compare against the strongest recent affective-generation systems cited in the paper and against simple baselines using exactly the same three-narrative augmentation—for example, sampling an accepted narrative description and prompting the same image generator, or retraining EmotiCrafter on the augmented data? This is needed to separate the contribution of the proposed decomposition/flow architecture from the contribution of additional model-generated supervision.

  3. Human validation and uncertainty: Can the authors provide human image-level evaluation of a representative, pre-specified subset for content satisfaction, narrative grouping, and perceived V/A, with annotator details, agreement statistics, confidence intervals, and agreement/correlation with the automated assessors? Do the method rankings and effect sizes remain similar under human judgments?

Reviewer_AMahOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies controllable affective image generation: given a content description and a target valence–arousal (V/A) value, the model generates multiple images that preserve protected content while varying visual narratives. It explicitly separates protected content from narrative elements, decomposes frozen SDXL text features into content and narrative representations, and uses content supervision plus reconstruction to enable content–narrative recombination. A conditional flow-matching model and an affect-conditioned transformer then generate and refine narrative prompts for the frozen SDXL. The authors extend the EmotiCrafter dataset and evaluate content preservation, affective validity, image quality, and key ablations.

Critical strengths
  1. The content/narrative split reframes a vague failure mode as a checkable constraint set, with protected content grounded in explicit spans and background/lighting/composition left free. The deterministic-predictor ablation cleanly isolates the mechanism claim: replacing flow sampling with an MSE regressor sharply reduces VNC while CSR remains nearly unchanged, showing that the sampling machinery does the work.

  2. The evaluation budget and judge protocol are well organized and honestly documented: all methods share descriptions, targets, seeds and generation budgets; the arithmetic is consistent; prompt sets are reproduced verbatim; and aggregation is conservative.

Critical weaknesses
  1. The paper claims a joint evaluation protocol, but the protocol is neither human-validated nor reproducible: all results come from an unpinned proprietary GPT-5.5-sol, no human or inter-assessor agreement is reported, and the same model also serves as a baseline. This leaves the headline evaluation claims without validity evidence and with an unmeasured potential bias.

  2. The most relevant methods (EmoGen, EmoCtrl, PromptMoG) are cited but absent from all comparisons, so the claimed benefit over existing content–emotion or prompt-sampling approaches is untested. Diversity is also confounded with validity: VNC rises mechanically with JVR, and LPIPS is computed on method-specific valid subsets.

Questions for authors
  1. If the three assessors are replaced by a non-OpenAI model family, or by human double-annotated images, how much do CSR/JVR/VNC move for each method? Please report judge–human agreement (κ) and the resulting shift in the Ours-versus-GPT-5.5+SDXL gap. Since the judge currently shares a model family with one baseline and has no fixed snapshot, this single number determines whether the reported ranking is a property of the methods or of the judge.

  2. Holding backbone, data and image seeds fixed, what is VNC if the flow-matching sampler is replaced by a PromptMoG-style MoG sampler in your own latent prompt space? The Deterministic ablation (Table 3) shows diversity does not come from image noise, but it does not show that a learned flow is necessary — a training-free mixture sampler is the missing control, and your own §2.3 already acknowledges the family resemblance.

  3. Conditional on conditions with at least two jointly valid outputs, what are VNC and LPIPS for all four methods, and what is the per-condition relation between n_valid and VNC?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.