acceptodds
Withdrawn from ICLR 2027

Content-Adaptive Aesthetic Alignment for Text-to-Image Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Aesthetic evaluation for text-to-image (T2I) generation must distinguish an inap- plicable criterion from a poorly realized one. To assess diverse visual content, we define a taxonomy of six aesthetic criteria, *i.e.*, realism, lighting, anatomy, creativ- ity, color harmony, and typography, and construct an aesthetic preference corpus of approximately 240K distinct images and 840K pairwise comparisons from hu- man and GPT-assisted criterion-specific rankings. Based on this corpus, we train **A**pplicability-Aware **W**eighted **A**esthetic **RE**ward (AWARE), which jointly pre- dicts quality and applicability for all six dimensions in one model. We then use AWARE to design an applicability-aware reinforcement-learning algorithm for text-to-image post-training, dynamically selecting and weighting only the crite- ria relevant to each sample. To evaluate both aesthetic judgment and post-trained generation, we introduce AWARE-RBench for criterion-conditioned preference and applicability evaluation, and AWARE-GenBench for role-conditional T2I aes- thetic quality. AWARE improves over the baseline by 60.92% on the aesthetic- model evaluation. Across Z-Image and SenseNova-U1.5, post-trained T2I models achieve average relative gains of 19.01%, 21.72%, and 2.39% on Photography, Artwork, and Infographic, respectively. On general-purpose benchmarks such as Qwen-Image-Bench, the aligned models remain competitive with existing open- source systems. Side-by-side visual comparisons further confirm improvements in fidelity, composition, natural lighting, and text rendering.

Primary area:
generative models
Keywords:
Text-to-image generation, Aesthetic Model, Applicability-aware RL training

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_YT7iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that aesthetic evaluation for text-to-image generation should distinguish a criterion that is inapplicable from one that is applicable but poorly realized. It introduces a role-conditioned taxonomy spanning Photography, Artwork, and Infographic content, centered on six criteria: realism, lighting, anatomy, creativity, color harmony, and typography. Based on approximately 240K images and 840K pairwise comparisons from human and GPT-assisted annotations, the authors train AWARE, a Qwen2-VL-7B-based reward model that predicts criterion-specific quality, uncertainty, and applicability. AWARE is then used in an applicability-aware RL objective that gates and weights criterion-level rewards for each prompt group. The paper also introduces AWARE-RBench for criterion-conditioned preference/applicability evaluation and AWARE-GenBench for role-conditioned generation evaluation. On the authors' benchmarks, AWARE reaches 87.54% pairwise accuracy, and AWARE-RL improves the reported role scores of Z-Image and SenseNova-U1.5-SFT. However, the paper does not isolate the contribution of learned applicability under a matched training setup, and the main evidence relies heavily on author-constructed benchmarks and GPT-based labels/evaluation whose key statistics and human validation are not reported in the submission.

Critical strengths
  1. Clear and consequential problem formulation. Separating criterion applicability from criterion quality directly addresses a real semantic failure of global aesthetic rewards. The distinction is actionable for both evaluation and training, and criterion-conflict pairs make the limitation of a single scalar score concrete.

  2. Ambitious end-to-end scope. The work connects a sizable preference corpus, a unified multi-criterion reward model, two diagnostic benchmarks, and RL experiments on two different generators. The reported gains on the authors' benchmarks and the use of both Z-Image and SenseNova-U1.5-SFT suggest that the idea is worth investigating beyond one model family.

Critical weaknesses
  1. The experiment does not isolate the paper's core mechanism. Table 2 compares unified applicability-aware training with role-specific experts, but those conditions differ in training-data composition and do not constitute a no-applicability control on the same mixed data and budget. The single-versus-multiple reward ablation likewise tests reward combination, not applicability. Without fixed-role, no-applicability, simple-router, and oracle-applicability baselines under matched data/compute, the observed gains could come from the 840K preference pairs, criterion-specific heads, mixed-domain training, or hand-designed role rewards rather than learned applicability. This directly undermines the title claim and the main scientific contribution.

  2. The evaluation is insufficiently independent and underdocumented. AWARE is trained and primarily evaluated within the authors' own taxonomy and data-generation/annotation pipeline; GPT-5.6-sol contributes labels and is also the sole AWARE-GenBench judge, merely sampled twice. The submission omits applicability/N/A prevalence, human-versus-GPT fractions, per-criterion agreement, adjudication rates, complete split counts, source audit, human calibration of GenBench. These are explicitly deferred to camera-ready. AWARE also trails HPSv3 on HPDv3 (69.56 versus 76.13), and RL reduces GenEval for both generators (73.37 to 68.38 for Z-Image; 92.86 to 90.10 for SenseNova), so external validity cannot be inferred from the reported in-house gains.

Questions for authors
  1. Under exactly matched prompts, number of samples, optimization steps, and compute, what are the results for (a) fixed role-conditioned criteria and weights without predicted applicability, (b) all six quality heads without applicability gating, (c) a simple role-label router, (d) oracle human applicability, and (e) predicted applicability? This comparison would materially determine whether the paper's central mechanism is responsible for the gains.

  2. How do approximately 330K five-candidate ranking tasks produce 840K pairwise examples over 240K distinct images? Please provide the exact group count, annotator-assignment count, pair-expansion rule, filtering attrition, and prompt/source/concept split audit. The current description does not allow the scale or leakage controls to be reconstructed.

  3. How well does AWARE-GenBench agree with independent blinded human judgments and with judges not used in data annotation? Please report paired uncertainty/significance for the 600 prompts and explain the substantial GenEval drops after RL. These results could change whether the claimed aesthetic gains represent general improvement or benchmark-specific optimization.

Reviewer_WDAyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to evaluate aesthetics for text-to-iamge generation. Diferent types of images require different criteria. Therefore, this paper defines six aesthetic criteria and build a corpus of 240K images. Based on the dataset, a reward model that predicts both quality and applicable probability for these six criteria. This reward model is used in the post-training for text-to-image models. The RL method makes full use of the reward model by dynamically selecting and weighting the criteria that matter for each sample. Experiments show clear gains on photography, artwork and infographics.

Critical strengths
  1. The motivation is clear. Content-adaptive alignment is a good choice. Different images and prompts should never be judged by the same fixed aesthetic rules.
  2. The research pipeline is complete and reasonable. This work forms a full loop from data collection to training and evaluation.
  3. The gains are large on the aesthetic evaluation and consistent across three content roles. Visual comparisons also support improvements in fidelity, lighting and text rendering.
Critical weaknesses
  1. The role of the applicability probability is not fully clarified. The paper does not clearly show the continuous probability provides fine-grained sample weighting. It may instead collapse into an implicit coarse category classifier, or in other words, act like a 0-1 gate for the three categories. Stronger evidence such as within category variance of applicability probability or an ablation comparing soft weighting with binarized gating will help.
  2. Novelty is limited at both the idea and method levels. The idea of content-adaptive weighting has been explored. For example, RubricRL (ECCV 2026) proposed prompt-adaptive weighting. Dynamic reward weighting is also commonly used.
Questions for authors

Please see weaknesses above.

Reviewer_E2u2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper performs aesthetic evaluation for text-to-image generation. Different from prior baselines, it measures images along six aesthetic criteria (realism, lighting, anatomy, creativity, color, typography) and disables criteria per group of images when they are deemed irrelevant for the majority of images in that group. Presumably, images are assigned to categories (photography, artwork, infographic) by the dataset they were obtained from - these are used as defaults to determine GT applicability labels and the annotator can adjust. The overall reward model design (including the VLM formulation) is built upon HPSv3 with an added head predicting applicability score. The applicability head is trained with cross entropy. Each criterion head gets its own pairwise ranking loss (applied only when the criteria is shared between the two images). The losses are then averaged together. The RL framework is Flow-GRPO, adapted to multi-reward following GDPO where each of the 6 criteria is a separate reward. The model is initialized from HPSv3 (with the head copied 6 times) and trained with LoRA. The model outperforms baselines on self-collected benchmarks but is worse on existing benchmarks.

Critical strengths

Multi-dimensional reward modeling for image generation is an interesting and valuable problem. Annotating criteria as N/A is a clean way to formulate it during the data collection stage. The multi-dimensional human preference dataset is also valuable. The design of Aware-RBench seems reasonable. The method improves results on its own Aware-GenBench on both ZImage and SenseNova.

Critical weaknesses

The method seems to only improve performance on AWARE-RBench while degrading scores on all other image reward benchmarks. According to Table 4, all other benchmarks (HPDv3, HPDv3++, ImageReward, PickScore) get much worse than HPSv3 baseline. Notably, HPDv3 has 57k real photos. Overall, the model seems to be training on data that is from the same annotation pipeline and distribution, potentially making Table 1 a bit of an unfair comparison. The method also degrades GenEval, as reported in Table 5 - 73.37->68.38 on Z-Image, 92.86->90.10 on SenseNova.

If Aware-GenBench is partially annotated by gpt-5.6-sol, and then reviewed by gpt-5.6-sol, that is circular evaluation. There doesn't seem to be human evaluation

The fact that the reward is not directly supervised for criteria which do not apply to an image seems problematic. A criterion is only trained if it is applicable for both images. So scores on images where a criterion doesn't apply can drift over the training. This then could affect the RL training dynamics depending on average anatomy score on images that don't contain a person. In the RL training if most of the images in the group show people, then my understanding is that the anatomy score for the remaining images that do not show people will still be incorporated into the group mean (and that score would not have been trained for those remaining images since it is not applicable to them). Depending on whether the anatomy scores for images that do not show people run consistently high or low, this could encourage either leaving people out, or penalizing cases that are not applicable for the anatomy reward. Besides, if much less than 50% of the images in each group have people, the anatomy reward is unlikely to trigger.

The exact algorithm for how GT category labels and applicability labels are obtained is not fully explained. To my best understanding, categories (photography, artwork, infographic) are assigned based on the dataset an image comes from and default applicability labels are assigned based on the category.

If it is true that applicability labels are assigned based on what dataset an image comes from, with only three distinct subsets (photography, artwork, infographic), then it is not very meaningful and the applicability labels would mainly just be category classification of the image - making them not really content adaptive? The error pattern in Table 6 suggests that the model labeling images as the wrong categories could explain much of the applicability errors across images, rather than having images being assigned independent applicability vectors. E.g. mistaking photography for artwork could flip realism/lighting/creativity/color harmony, which all have very similar errors around 1%. And the fact that infographic gets 100% applicability classification accuracy suggests that infographic classification is very easy and that infographics might not have other difficult-to-classify applicability labels mixed in.

The impact of in-domain data is not directly ablated (e.g. one could imagine that applicability is only collected during human/LLM data annotation and then merged into a single pairwise preference score which is used to directly fine-tune HPSv3 without architecture changes).

HPSv3++'s reported results on HPDv3 in Table 4 (76.39) seem significantly worse than their own reported benchmark results in HPSv3++'s Table 2 (86.7)

The writing is quite sloppy especially in the appendix - many things are promised but not discussed. There is potential LLM generated template text in lines 755-758: "We report role and criterion distributions, applicable/N/A ratios, the human and GPT-assisted fraction, repeated and adjudicated item counts, and split counts. These quantities are kept as editable commands in the main file so that the paper can be updated after the final data audit." Section A.4 is missing all of its content. Section A.10 promises class imbalance and applicability variants that are not disclosed. Etc.

Questions for authors

As above.

How is this model supposed to be used on images at test time where a criterion applies to one image but not the other.

How category and applicability labels are obtained and whether applicability labels are fixed per category / statistics on applicability label distribution per category (e.g. for each category, what is the average applicability label within that category).

In Table 1, is every pair in the benchmark scored on exactly one criterion? How many of the 1750 annotated image pairs belong to each criterion.

How is the "separate overall aesthetic preference label" on L287 computed.

Some discussion on the training dynamics of the RL, particularly on cases where the criterion doesn't apply to some portion of the images in the group, as well as statistics of the applicability scores within groups, would be appreciated

Reviewer_oqhZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper points out that global aesthetic reward models treat "this criterion doesn't apply" the same as "this criterion is done badly". It defines six criteria (realism, lighting, anatomy, creativity, color harmony, typography) over three content types (Photography, Artwork, Infographic), and builds a preference dataset of about 240K images and 840K pairs from human and GPT-assisted rankings. On this data the authors train AWARE, a reward model initialized from HPSv3 that predicts per-criterion quality (with uncertainty) and whether each criterion applies, all in one forward pass. AWARE is then used in a GRPO-style RL objective where criteria are turned on and weighted by their average applicability within the group. The paper also introduces two benchmarks: AWARE-RBench for criterion-level preference and applicability, and AWARE-GenBench for generation quality per content type. AWARE gets 87.5% on RBench versus 54.4% for HPSv3, and RL improves Z-Image and SenseNova-U1.5 on GenBench.

Critical strengths
  • Separating applicability from quality is well motivated and useful in practice for T2I models that generate many kinds of content.
  • The dataset is large and has criterion-specific human rankings over real-real, real-generated and generated-generated pairs. It would be a good resource if released.
  • The hard pairs and criterion-conflict pairs in RBench are a good way to test criterion-specific judgment.
  • The approach is simple and cheap during RL, since one forward pass gives all criterion rewards.
Critical weaknesses
  1. The evaluation is circular. Both main evaluations use benchmarks the authors built with the same pipeline as the training data. GenBench is scored by GPT-5.6-sol, which was also used to help annotate the training data, so a generator tuned against AWARE may be rewarded by a judge whose preferences were partly baked into AWARE. There is no human evaluation of the generated images. On the existing external benchmarks (Table 4), AWARE is clearly worse than HPSv3, the model it starts from (HPDv3 69.6 vs 76.1, ImageReward 58.6 vs 67.2, PickScore 64.6 vs 69.7). The authors put this down to low-quality older pairs, but it could just as well mean AWARE is fitting the authors' own annotation style.

  2. The ablation that matters most is missing. The main claim is that predicted applicability is what makes content-adaptive alignment work. But applicability accuracy is 97-100%, and it is almost entirely determined by the content type (Table 6 shows 100% on Infographic). The obvious baseline is the same multi-head reward with a fixed mask per content type, or just an equal-weight sum of all six heads. There is also no RL comparison with existing rewards such as HPSv3, VisionReward, or multi-reward GDPO with fixed weights. Without these, I can't tell whether the gains come from applicability, from new reward heads trained on new data, or just from the extra OCR reward on infographics.

  3. Some numbers don't match. In Table 7 (appendix), the single- and two-reward Z-Image runs reach 75.22 on Photography (Lighting+Realism) and 47.83 on Infographic (OCR only). Both are higher than full AWARE-RL on Z-Image in Table 2 (67.25 and 40.33). Is this a different RL budget, a different subset, or a different checkpoint? As it stands, it looks like the unified objective does worse than simpler per-type reward combinations.

  4. The generation gains are modest and there are regressions. The Infographic gain is small (+1.36% on Z-Image). For Z-Image, GenEval drops from 73.37 to 68.38 and QIB also goes down a bit (Table 5). Reward hacking is not studied, apart from a qualitative note that Lighting alone causes overexposure. The "60.92% improvement" in the abstract is a relative gain over an HPSv3 baseline that is close to chance on this benchmark, so it overstates things.

  5. The comparison with VLM judges looks unfair. The Hard protocol (counted correct only if all three orderings agree) is applied to the proprietary VLMs, but as far as I can tell not to AWARE or the other reward models. This pulls down the strongest competitors, e.g. their anatomy accuracy falls below 50%.

  6. Reproducibility is a concern. Appendix A.3-A.4 says the data statistics are "kept as editable commands in the main file so that the paper can be updated after the final data audit". The annotation interface, rubric, GPT prompts, agreement statistics, adjudication rates and split audit are all deferred to the camera-ready. Since the dataset, reward model and benchmarks are the main contributions, reviewers need this information now. The share of human versus GPT labels, the per-criterion annotator agreement, and the train/test split sizes are not given.

  7. The method itself is not very new. It combines HPSv3's uncertainty-aware pairwise loss, multiple heads, a BCE applicability head, GDPO-style per-criterion normalization and Flow-GRPO. The applicability gating is a threshold on a group average. Most of the contribution is in the data and engineering.

Questions for authors
  1. Why do the simpler reward combinations in Table 7 score higher than full AWARE-RL in Table 2 on Photography and Infographic?
  2. What happens if predicted applicability is replaced with a fixed per-type mask, or if all six criteria are used with equal weights?
  3. Could you add a human evaluation of generated images, or use a GenBench judge that was not involved in labeling the training data?
  4. Was the Hard protocol used for AWARE and the other reward models too? If not, could you report everything under the same protocol?
  5. Please give per-criterion annotator agreement, the fraction of GPT labels, and a check for overlap between train and test, at least at the prompt and source level.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.