acceptodds
Under review as a conference paper at ICLR 2027

AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment

Abstract

Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment and perceptual quality. Existing reward models commonly require training on large-scale human preference corpora. Meanwhile, more recent work leverages Vision-Language Model (VLM) judges with rubrics to make evaluation criteria explicit. However, rubric-based approaches often lack reliable methods for criterion selection. In this paper, we propose , a framework that automatically synthesizes, selects, and weights explicit rubrics to guide VLM judges. AutoRubric-T2I first extracts candidate rubrics from reasoning traces over preference pairs and scores the paired images under each rubric using a frozen VLM judge. Then, an \textbf{\ell_1-Regularized Logistic Regression Refiner} fits these scores to human preference labels, pruning rubrics that fail to guide the judge toward preference-consistent assessments and learning dataset-specific weights for the retained criteria. Using just 256 preference pairs (0.02-0.04% of full RM training data) during training, AutoRubric-T2I outperforms fine-tuned reward models on MMRB2. As an RL reward in Flow-GRPO, it also improves generation quality over the fine-tuned reward models on TIIF and UniGenBench++.

Primary area:
generative models
Keywords:
Reward Model, Text-to-Image Alignment, Vision Language Model, RLHF

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XDdYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AutoRubric-T2I learns an explicit reward function for text-to-image alignment from image preference pairs. A frozen VLM scores images against generated rubrics, and L1-regularized logistic regression selects and weights rubric features to predict preferences. Iterative refinement expands the pool using misranked pairs selected by a curriculum. The resulting weighted pointwise reward is evaluated on preference benchmarks and used to fine-tune SD3.5-Medium with Flow-GRPO. Experiments with three judges, component ablations, and downstream evaluations report gains over the implemented reward-model and rubric baselines.

Critical strengths
  1. A coherent approach to a meaningful problem. Preference-supervised selection and weighting directly address the usefulness of individual rubrics, while retaining explicit criteria and a frozen judge. Updating the pool from the current evaluator's errors provides a concrete mechanism for improving its coverage.
  2. Useful empirical support across evaluation and optimization. The paper evaluates three judges and demonstrates downstream improvements on TIIF and UniGenBench++. Component ablations and cost measurements help assess the practical value of the design beyond preference prediction alone.
Critical weaknesses
  1. The comparison does not clearly establish the benefit of the proposed scoring design. Assessing sparse weighting and rubric refinement requires an effective rubric baseline. Yet Figure 3 and Table 2 adapt Auto-Rubric's pairwise procedure to pointwise evaluation without adequately explaining the adaptation. This makes it difficult to attribute the performance gap to the proposed learning mechanism. The choice of preference accuracy further limits the conclusion: it evaluates ranking, which either scoring mode can support, and does not independently establish an advantage of pointwise scoring. The results therefore leave the benefit of the scoring design insufficiently established.
  2. Unclear model selection weakens the evidence for transfer and downstream improvement. The limited-supervision and transfer claims depend on which labels are used to select the rubric set. Appendix G (p.19, L1006–1008) specifies 256 validation pairs from the other source corpus, while Appendix H (p.20, L1039–1043) specifies the same source. This leaves the supervision budget and transfer setting unclear. Downstream selection is also incompletely reported: Appendix B gives a training range and evaluation interval, but not the checkpoint-selection rule for Figure 4 and Tables 3–4. This matters for gains as small as 0.4–0.7 percentage points over AutoRule. Independent support from the human study is limited: its 600 judgments cover only 20 prompts, with no selection rationale.
Questions for authors
  1. Which validation source was used for each result? Please reconcile Appendices G and H, specify all labels used for model selection, and identify the checkpoint-selection rule for Figure 4 and Tables 3–4.
  2. How was Auto-Rubric adapted to pointwise evaluation, and what evidence shows that the adaptation preserves its effectiveness? Please also reconcile the conflicting descriptions of RubricRL's offline caching and generation during RL in §2.3 and Appendix C.
  3. Some of the existing methods such as ARR [1] also develop visual preference rubrics through generation, verification, and refinement for evaluation and downstream optimization. Discussing their relationship to the learned global weights and curriculum mining here would help position the contribution.

[1] Juanxi Tian et al. Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria. arXiv preprint arXiv:2605.08354v1. 2026.

1 comment

Thank you for the detailed review. In short: Table 2 uses Auto-Rubric's native pointwise mode, and pairwise re-evaluation confirms our rubrics still lead by 3.3–8.1 points across all benchmarks; all results use cross-dataset validation and a fixed step-500 checkpoint; and we will fix Appendices C, G, and H, include the new RL reward curves in the main text, add prompt-level bootstrap CIs for the human study, and discuss ARR [1].

W1. Scoring design and Auto-Rubric baseline

How Auto-Rubric was run. Our comparison tests how rubrics are learned (sparse positive weights + hard-pair mining), not whether pointwise beats pairwise scoring. Although the Auto-Rubric paper describes a pairwise protocol, its OpenJudge code defaults to pointwise grading (grader_mode = GraderMode.POINTWISE in openjudge/generator/llm_grader_generator.py, commit 4dcb30d). Table 2 thus uses Auto-Rubric's native implementation, passing image pairs and labels to its generator as in the paper (clarified in Appendix D).

Pairwise verification. Rerunning both methods under pairwise scoring with the same prompt (averaging both orders) confirms pointwise mode did not disadvantage Auto-Rubric:

Qwen3-VL-8B, MMRB2 accuracy (%)Pointwise (Table 2)Pairwise (new)
Judge alone, no rubrics58.0*59.4
Auto-Rubric (HPDv3 / Pick-a-Pic)57.5 / 57.060.7 / 60.3
AutoRubric-T2I (HPDv3 / Pick-a-Pic)62.5 / 62.464.6 / 64.3

*Tie-adjusted as in Table 2 (26.5 if ties count as errors).

On MMRB2, Auto-Rubric shifts judge accuracy by ≤1.3\le 1.3 points (lowering it pointwise), while ours adds 4.4–5.2 points in both modes. Pairwise Auto-Rubric also drops below the judge alone on HPDv3 and Pick-a-Pic, whereas we lead Auto-Rubric across all three benchmarks by 4.4–8.4 points pointwise and 3.3–8.1 points pairwise.

Why pointwise scoring. Pairwise judging is easier (1.9–4.2 points higher with our rubrics) as it sees both images together. We use pointwise scoring because Flow-GRPO normalizes rewards across K=24K=24 rollouts (Appendix B), needing 24 judge calls per group vs. 276 pairs (552 order-balanced). Its utility is shown in Figure 4, beating scalar rewards in all eight settings.


W2. Model selection, checkpoints, and human study

Validation labels (Appendices G & H). All results use cross-dataset validation: rubrics trained on 256 HPDv3 pairs pick their round on 256 Pick-a-Pic pairs, and vice versa. These 256 pairs are the sole selection labels (never entering generation or weight fitting), totaling 512 pairs per run with zero MMRB2 overlap. We will fix the same-dataset split in Appendix H and ambiguous text in Appendix G, and state the 256 + 256 budget in the main text.

Checkpoint rule. Figure 4 and Tables 3–4 evaluate the fixed step-500 checkpoint without benchmark selection. By step 500, runs reach 85–95% of their 800-step max reward, while longer training induces reward hacking (e.g., HPSv3 destabilizes around step 700). We will include these new reward curves in the main text and state the step-500 rule in Appendix B. Across three seeds (see Reviewer 2), ours leads TIIF mean in all four settings (69.0 overall vs. 67.4 AutoRule and 66.4 scalar), though single-setting gaps to AutoRule fall within seed spread.

Human study. The 20 prompts were uniformly sampled from UniGenBench and TIIF, showing four anonymized step-500 images in random order. Treating this as supporting evidence and bootstrapping over prompts for prompt-level correlation, AutoRubric-T2I is chosen 44.8% of the time (95% CI: [39.6%, 46.7%]). We will add the sampling rule and CI to the revision.


Q1 & Q2. Validation, checkpoints, Auto-Rubric, and RubricRL

Validation labels, checkpoints, and Auto-Rubric's pointwise mode are covered in W1–W2. For RubricRL, §2.3 is accurate and Appendix C will be fixed: RubricRL caches prompt-specific checklists offline and runs only scoring during RL (Appendix C followed an earlier arXiv draft). Unlike RubricRL's per-prompt, equally weighted checklists, we learn one shared rubric set with sparse fitted weights.


Q3. Relation to ARR [1]

We will cite and discuss concurrent work ARR [1] in Section 2 and Appendix C. Both refine visual rubrics from small preference sets, but ARR merges verified rubrics into one structured prompt for pairwise win/loss decisions without per-rubric weights. We score each rubric independently, fit sparse positive global weights (+1.2 points on MMRB2), and mine misranked pairs (+1.3 points over random-pair refinement, Figure 5). On Qwen3-VL-8B MMRB2, ARR reports 62.7% (+5.1 over its pairwise judge), comparable to our pairwise accuracy of 64.3–64.6% (+4.9–5.2 over ours), while our per-rubric formulation also serves as a pointwise RL reward.

[1] Juanxi Tian et al. Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria. arXiv:2605.08354v1, 2026.

Reviewer_ibsKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AutoRubric-T2I, a rubric-based reward model for text-to-image generation that is learned from only a small number of preference pairs. Candidate rubrics are generated with a VLM and then selected and weighted using L1-regularized logistic regression. The method also uses misranked preference pairs to generate new rubrics over several refinement rounds. The final rubric set is evaluated on MMRB2 and is also used as the reward in Flow-GRPO, where the authors report improvements on TIIF and UniGenBench++.

Critical strengths

I think the explicit reward is the strongest part of the paper. The final score is a weighted combination of individual rubric scores, so it is much easier to inspect what the reward is actually using. I also like that the VLM judge remains frozen instead of being fine-tuned for this task.

The downstream RL experiments are important here. Without them, the work would mainly be another way of evaluating image preferences. Using the learned reward in Flow-GRPO and seeing improvements on TIIF and UniGenBench++ gives stronger evidence that the rubric reward is useful for training as well.

Critical weaknesses

I am not fully convinced by the size of some of the gains. For example, with Gemini-3-Flash, direct pairwise judging gives 70.8% on MMRB2, while AutoRubric-T2I gives 71.1%. The difference is quite small, so it is hard to tell from the current results how much of the gain comes from the proposed rubric learning itself and how much may come from run-to-run variation.

There is also a fairly large inference cost during RL. The final reward evaluates 20 rubrics for each generated image, so the reward requires 20 VLM evaluations per image. This is very different from a conventional scalar reward model. The paper does discuss this trade-off, but I think it is important when considering the practical benefit of using only 256 preference pairs, since the annotation cost is reduced at the expense of substantially higher reward computation during RL.

Questions for authors
  1. Could you clarify the validation protocol described in Appendices G and H? In particular, which data are used to select the best refinement round, and how many labeled preference pairs are used for model selection in total?
  2. Do you have results from multiple runs for the rubric-learning and Flow-GRPO experiments? It would be helpful to know whether the reported improvements are stable, especially for the smaller gains.
1 comment

Thank you for the constructive review. In short: (1) 70.8% vs. 71.1% compares two scoring modes, whereas with mode fixed, our rubrics lift pointwise Gemini-3-Flash by +9.0 points (62.1% →\rightarrow 71.1%); (2) rubric-learning std is ≤0.3\le 0.3 points on MMRB2 and 3-seed RL keeps the same ordering; (3) vLLM prefix caching adds ~12% per RL iteration.

W1. Size of the gains (70.8% vs. 71.1%) and scoring modes

Comparing 70.8% against 71.1% directly is unfair because they use different scoring mechanisms. Even zero-shot without rubrics, every VLM shows a large gap between pointwise and pairwise judging since pairwise sees both images together: in Table 2, zero-shot Gemini-3-Flash scores 70.8% pairwise vs. 62.1% pointwise (tie-adjusted)—an 8.7-point gap (similar on Qwen3-VL-8B). Our rubrics raise pointwise Gemini-3-Flash by 9.0 points to 71.1% on average, overcoming the entire zero-shot modality gap while seeing one image at a time.

With scoring mode fixed, gains hold across modes:

  • Pointwise, Gemini-3-Flash. Zero-shot reaches 62.1% on MMRB2; our rubrics reach 70.8% (HPDv3) and 71.4% (Pick-a-Pic), averaging 71.1%, vs. 67.6% and 64.7% for AutoRule under the same judge, format, and 20-rubric budget.
  • Pairwise, Gemini-3-Flash. Keeping our Pick-a-Pic rubrics unchanged and switching only the final scoring mode to pairwise lifts accuracy from 71.4% to 77.2% (+6.4 points over the 70.8% pairwise judge).
  • Pairwise, Qwen3-VL-8B (see Reviewer XDdY). Zero-shot pairwise reaches 59.4%; our rubrics scored pairwise reach 64.6% and 64.3% (~+5 points), vs. 60.7% and 60.3% for AutoRule.

We use pointwise scoring in RL because pairwise judging gives no standalone per-rollout score; using it in GRPO requires preference-to-reward conversion and position-bias correction [1]. Our rubrics give a direct per-image score, and rubric-learning std (≤0.3\le 0.3 points, Q2) is an order of magnitude smaller than these 3–9 point gains.


W2. Reward cost during RL

The 20 rubric queries share the image, instructions, and prompt prefix, differing only in the ~146-token rubric suffix. With vLLM prefix caching, this prefix is encoded once per image with identical outputs up to numerical precision (Qwen3-VL-8B bf16, Flow-GRPO SD3.5-Medium on 8$\times$ H200, 1,152 images/iter):

RewardLatency (ms/img, 1$\times$ H200)Iter time (s)
HPSv331.567.4
Single rubric74.867.0
20 rubrics, independent560.0103.8
20 rubrics, shared prefix74.275.4

With prefix caching, per-image latency matches a single rubric as the 20 suffixes run in parallel, adding 12% to iteration time over HPSv3 (~33 min over 500 steps / 250 iters). Using only the top-4 weight-ranked rubrics matches HPSv3 compute while keeping downstream accuracy (68.4 vs. 68.3 on Pick-a-Pic; -0.5 points on HPDv3, Table 9). Each run uses 512 pairs total (Q1)—0.04–0.08% of HPSv3/PickScore training data—and a one-time 2–4h rubric-learning step on 4$\times$ A6000 GPUs without reward-model training.


Q1. Validation protocol and labeled pairs

The best refinement round is chosen cross-corpus: rubrics trained on Pick-a-Pic validate on 256 labeled HPDv3 pairs, and vice versa. These 256 pairs are fixed across all ten rounds and used solely to pick the round (never in rubric generation or weight fitting), totaling 256 + 256 = 512 pairs per run. Cross-corpus validation favors out-of-distribution generalization tested on MMRB2 (zero overlap).


Q2. Multiple runs of rubric learning and Flow-GRPO

Rubric learning (3 runs). With temperature-0 judging and deterministic seed selection, varying rubric generation (Gemini-3-Flash, temperature 0.1) across three reruns gives Qwen3-VL-8B MMRB2 accuracy of 62.4 ±\pm 0.3 (HPDv3) and 62.6 ±\pm 0.2 (Pick-a-Pic), with in-domain std ≤0.6\le 0.6. Sampling 256 seed pairs from three disjoint pools gives 62.3 ±\pm 0.2 and 62.2 ±\pm 0.3—an order of magnitude smaller than the W1 gains.

Flow-GRPO (3 seeds). Training SD3.5-Medium for 500 steps (final checkpoint, no selection), TIIF scores (mean ±\pm std) are:

RewardPromptsScalarAutoRuleOurs
HPSv3Short68.3 ±\pm 1.568.9 ±\pm 3.970.5 ±\pm 4.0
HPSv3Long63.9 ±\pm 2.766.6 ±\pm 0.667.2 ±\pm 4.7
PickScoreShort66.8 ±\pm 2.568.0 ±\pm 5.569.3 ±\pm 3.5
PickScoreLong66.7 ±\pm 3.166.0 ±\pm 0.469.1 ±\pm 1.2

Ours leads in all four settings (+2.2–3.3 over scalar; +0.6–3.1 over AutoRule), with overall mean 69.0 vs. 67.4 (AutoRule) and 66.4 (scalar). While seed spread (0.4–5.5) overlaps within single settings, consistent ordering across all reward sources and prompt lengths confirms the gain.

[1] Zhu et al. TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design. arXiv:2605.20731, 2026.

Reviewer_QgMCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AutoRubric-T2I, a framework for learning a compact set of weighted, global rubrics from human preference data for text-to-image (T2I) reward modeling. The main motivation is to address the limitations of existing reward models, which often require large amounts of preference data and lack interpretability, as well as existing rubric-based methods that may struggle to select effective evaluation criteria and capture human preferences.

Specifically, the proposed method first generates candidate rubrics from preference pairs using a VLM, and then employs L1-regularized logistic regression to select informative rubrics and learn their weights. It further incorporates hard-pair mining and iterative rubric refinement to improve the quality of the learned global rubric set. The resulting weighted rubrics can be used with a frozen VLM judge to provide reward signals without additional model fine-tuning.

The authors evaluate their approach on preference prediction benchmarks, including MMRB2, HPDv3, and Pick-a-Pic, and further validate its effectiveness in downstream T2I reinforcement learning using Flow-GRPO on TIIF and UniGenBench++. The experimental results show improvements over several existing reward models and rubric-based baselines, particularly in out-of-domain preference prediction and downstream image generation. The paper also includes ablation studies and human evaluations to support the effectiveness of the proposed framework.

Critical strengths
  1. Clear and Well-Motivated Problem Formulation. The paper addresses a meaningful problem in T2I reward modeling. Existing reward models often require substantial preference data and lack interpretability, while rubric-based approaches face challenges in selecting effective evaluation criteria. I think the authors clearly identify these limitations and provide a reasonable motivation for learning preference-aligned rubrics automatically.

  2. Interesting and Reasonable Method Design. I find the overall framework interesting and quite intuitive. In particular, combining sparse rubric selection with hard-pair mining and iterative refinement is a reasonable way to identify informative evaluation criteria. I also like the idea of learning a compact set of weighted global rubrics from human preferences, rather than relying on manually designed or equally weighted rules. This provides a potentially useful direction for building more interpretable and data-efficient reward models.

Critical weaknesses
  1. Limited and Less Convincing Qualitative Results. Since this paper focuses on text-to-image generation, I think it is important to include some qualitative comparisons in the main paper rather than placing all the generated examples in the appendix. More importantly, I am not fully convinced by some of the visual results in the appendix. For example, in Figure 11, the second, third, and fifth examples do not seem particularly strong to me. Some generated images still appear to have issues with text alignment or contain unnatural visual details. I think the authors should provide more convincing qualitative results to better demonstrate the effectiveness of the proposed reward model, especially in terms of visual quality and prompt alignment.

  2. Limited Technical Novelty. Although I find the overall framework interesting and reasonable, I am not entirely convinced about its technical novelty. Most of the key components, including rubric generation, L1-regularized logistic regression, and hard-pair mining, are based on existing techniques. To me, the proposed method feels more like a well-designed engineering framework that combines existing ideas, rather than introducing a fundamentally new algorithmic contribution. I would like the authors to better clarify what the core technical innovation is and how it differs from existing rubric-based approaches.

Questions for authors
  1. Qualitative Results. Since this paper focuses on T2I generation, could the authors provide more convincing qualitative comparisons? In particular, some examples in Figure 11 (e.g., the second, third, and fifth examples) still seem to have issues with prompt alignment and visual quality. Are these representative of the overall performance? I think more qualitative evidence would help me better understand the actual improvements brought by the proposed method.

  2. Core Technical Novelty. Could the authors further clarify the main technical novelty of the proposed framework? Since rubric generation, L1-regularized logistic regression, and hard-pair mining are all established techniques, I would like to better understand what the key innovation is beyond combining these existing components.

  3. Stability of the Learned Global Rubrics. I am particularly curious about the stability of the learned global rubric set. Since the rubrics are learned from only 256 preference pairs, I would expect that sampling different subsets from the same dataset might lead to different global rubrics and weights. Have the authors investigated how consistent the learned rubrics are across different random subsets or seeds? More importantly, even if the learned rubrics differ, do they produce consistent evaluation results? I think this is important for understanding the robustness and reliability of the proposed method.

  4. Subjective Preference Modeling. Finally, I wonder whether the learned global rubrics can effectively capture subjective human preferences, such as aesthetics and overall visual appeal, rather than mainly improving evaluation on relatively objective criteria like object counts, attributes, and spatial relationships. Could the authors provide more analysis or evidence on this aspect?

4 comments

Thank you for the constructive review. Unlike prior rubric methods, AutoRubric-T2I uses preference labels both to select which rubrics to keep and to learn their weights, making it the only rubric method that outperforms the same VLM used as a direct pairwise judge on MMRB2 across all three judges.

W2 & Q2. Novelty and problem formulation

Our contribution is the formulation rather than individual components. We frame rubric learning as sparse logistic regression over the infinite space of natural-language rubrics (Eq. 1) and solve it with a working-set method: the LLM proposes candidate coordinates, the ℓ1\ell_1 refiner selects and weights them against preference labels, and the errors of the current weighted set drive the next proposals. Prior rubric methods do not use preference labels to select and weight rubrics—AutoRule and RubricRL average rubrics uniformly, while Auto-Rubric selects rubrics by semantic diversity rather than predictive fit to human preferences.

Two results show that this is not a simple combination of known parts:

  • Component isolation (Figure 5). Naively adding an ℓ1\ell_1 refiner to AutoRule changes MMRB2 accuracy by only +0.2 points. The gains come from the positivity constraint (+1.2 points) and error-driven hard-pair mining (+1.3 points beyond random-pair refinement).
  • Outperforming the pairwise judge while remaining pointwise. With Gemini-3-Flash and the same 20-rubric budget, AutoRule and Auto-Rubric fall below the direct pairwise judge (66.2% and 61.2% vs. 70.8%). AutoRubric-T2I exceeds it across all three VLMs (71.1%, 66.6%, and 62.5%) while staying pointwise, enabling direct use as a scalar RL reward.

W1 & Q1. Qualitative results

The qualitative figures were placed in the appendix due to page limits; we will include the new side-by-side qualitative comparisons directly in the main text and add further examples to the appendix. We agree that the examples in Figure 11 are not flawless. Our claim is about relative improvement over baselines and rests on aggregate evidence rather than individual images:

  • On TIIF and UniGenBench++, AutoRubric-T2I outperforms the matching scalar reward by 1.4–4.3 points across all eight settings (Figure 4, Tables 3–4).
  • In a 4-way human study (20 prompts, 30 annotators), our method is chosen 44.8% of the time against a 25% chance rate (95% bootstrap CI over prompts: [39.6%, 46.7%]).
  • Figure 9 shows that scalar-reward policies drift toward degraded, over-stylized outputs as reward increases, whereas our policies remain natural.

Q3. Stability across seed pools and runs

We ran the exact experiment suggested. For each dataset, we split the training set into three disjoint pools, applied the same diversity-aware selection within each pool to sample 256 seed pairs, and ran the full pipeline on each. We also reran the pipeline three times on the original seed pairs, varying only rubric generation (Gemini-3-Flash, temperature 0.1) and the refinement rounds depending on it:

Qwen3-VL-8B, accuracy (%), mean ±\pm std over 3 runsMMRB2Pick-a-PicHPDv3
HPDv3 rubrics, same seed pairs62.4 ±\pm 0.361.8 ±\pm 0.563.7 ±\pm 0.6
HPDv3 rubrics, 3 disjoint pools62.3 ±\pm 0.261.5 ±\pm 0.463.6 ±\pm 0.3
Pick-a-Pic rubrics, same seed pairs62.6 ±\pm 0.263.5 ±\pm 0.361.9 ±\pm 0.5
Pick-a-Pic rubrics, 3 disjoint pools62.2 ±\pm 0.362.8 ±\pm 0.761.1 ±\pm 0.5

Drawing seed pairs from disjoint pools varies results no more than rerunning on the same pairs: MMRB2 standard deviation is at most 0.3 points in both settings. Every run stays well above the direct pairwise judge (59.4%) and AutoRule (59.1% and 56.4%). Although rubrics learned from HPDv3 and Pick-a-Pic differ textually (Appendix O), they agree within 0.1 to 2.3 points across all three judges (Table 2).


Q4. Subjective and aesthetic preferences

Our rubrics are fit to holistic human preference labels rather than object-level annotations. In Figure 17, the top-weighted rubric captures lighting, color palette, and atmosphere, and another explicitly favors compositional impact over literal accuracy (such as object counts). Adding a specialized aesthetic model's score as an extra regression feature yields negligible weight, indicating it adds little signal beyond the rubrics.

Downstream, the LAION aesthetic predictor (never used as a reward) improves over the SD3.5-Medium base (5.840) by 0.06–0.13 points for every rubric-RL variant on 1,200 UniGenBench images (paired p<10−15p < 10^{-15}; 59–68% of images improve). Because a preliminary prompt-blind human comparison against the base model is mixed (win rates 58.8% and 43.9% across two runs), we claim no aesthetic degradation rather than a consistent aesthetic improvement.

Thank you for conducting the additional experiments and providing detailed responses to my questions. Your clarifications have addressed my other concerns, and I have no further questions regarding those aspects.

However, I still have some reservations about the qualitative results. I understand that the quantitative metrics demonstrate consistent improvements, but based on the visual examples provided so far, I personally find it difficult to clearly perceive the advantages of the proposed method over the baselines.

Since you have already conducted a human evaluation, would it be possible to provide around 10 representative examples (i.e., prompt-level comparisons) from the human study, particularly those in which human annotators showed relatively strong agreement in favor of your method? It would be helpful to see the corresponding generated images side by side, so that I can better understand whether the improvements reflected in the quantitative results are also visually perceptible to human observers.

I appreciate your efforts in addressing my concerns, and I believe these additional qualitative examples would further strengthen the paper.

One small, lighthearted suggestion for future rounds: it would be great if the response could come a little earlier, as that would leave us more time for discussion! :)

Thank you for your prompt follow-up and encouraging feedback—and noted on the timing for future rounds! :)

Following your suggestion, we have updated the manuscript with 10 side-by-side 4-way comparisons (Base, Scalar Reward, AutoRule, and AutoRubric-T2I) selected directly by ranking all 20 human-study prompts by annotator agreement in favor of our method (Figure 17 in Appendix N.1 shows the top-5 prompts, and Figure 7 shows the additional quantitative results in the revised PDF).

To make the perceptual differences easier to inspect alongside the quantitative preference data, each row is annotated with its exact 30-annotator vote breakdown:

  1. Strong annotator consensus across top prompts: In our 4-way forced-choice setting (25% random baseline), the top-5 prompts achieve 50.0%–70.0% agreement (15/30 to 21/30 annotators), and ranks 6–10 achieve 46.7%–33.3% (14/30 to 10/30):

    • Rank 1 (#20, "Green ball vs. Red ball"): 70.0% (21/30 votes for AutoRubric-T2I)
    • Rank 2 (#17, "Pillow on armchair"): 63.3% (19/30 votes)
    • Rank 3 (#1, "Table in front of painting"): 60.0% (18/30 votes)
    • Rank 4 (#4, "Cat beneath table"): 50.0% (15/30 votes)
    • Rank 5 (#7, "Tourist attraction travel journal"): 50.0% (15/30 votes) (Across all 20 prompts, AutoRubric-T2I wins the plurality vote on 10/20 prompts, confirming these examples reflect a consistent trend.)
  2. What human annotators consistently prefer in these comparisons:

    • Cleaner spatial layout and attribute binding without sacrificing visual polish (#20, #17, #1, #4): Base and AutoRule frequently misplace relative object positions (e.g., in front of, beneath, on) or confuse object-color bindings, whereas AutoRubric-T2I satisfies the prompt constraints with natural object proportions and coherent backgrounds.
    • Natural lighting and compositional harmony vs. Scalar Rewards (#17, #1, #7): Scalar-reward policies often push contrast, saturation, or local textures to unnatural extremes, whereas AutoRubric-T2I preserves realistic lighting, material textures, and balanced layout on complex scenes such as the travel journal (#7).

We have uploaded the revised PDF with these comparisons and vote breakdowns, and hope they make the visual improvements clearer. Please let us know if any further details would be helpful!

Thanks for the update and all your efforts. No worries at all about the timing. We'll have a discussion further in the upcoming discussion phase.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Rebuttal

We thank all reviewers for their constructive feedback and recognition of our rubric-learning formulation and downstream RL gains. Across the reviews, we added new experiments on scoring-mode fairness, multi-run and multi-seed stability, and RL inference efficiency, and clarified our validation and checkpoint protocols. Below is a summary of our responses and planned revisions:

Reviewer & ConcernRebuttal ResponseChanges in the Paper
R1 (W2/Q2): Novelty & formulation vs. prior rubric methodsFramed as sparse logistic regression solved via a working-set method. Ablations isolate positive weights (+1.2) and hard-pair mining (+1.3) vs. naive ℓ1\ell_1 (+0.2). Only rubric method beating the direct pairwise judge across all 3 VLMs while staying pointwise.Sharpen the formulation contrast against AutoRule, Auto-Rubric, and RubricRL in Section 2.
R1 (W1/Q1/Q4): Qualitative results & aesthetic qualityAggregate gains hold across all 8 RL settings (+1.4–4.3), human study (44.8% vs. 25% chance), and LAION aesthetic scores (+0.06–0.13, p<10−15p<10^{-15}). Blind aesthetic win rates are mixed (58.8% / 43.9%), supporting no degradation.Move new side-by-side qualitative comparisons into the main text; add extra examples and conservative aesthetic claims in the appendix.
R1 (Q3) & R2 (Q2): Stability across seed pools & RL seeds3-run rubric learning and 3 disjoint seed pools both yield MMRB2 std ≤0.3\le 0.3 points (Qwen3-VL-8B). 3-seed Flow-GRPO leads TIIF mean in all 4 settings (69.0 overall vs. 67.4 AutoRule, 66.4 scalar).Add multi-run rubric learning (same seeds & disjoint pools) and 3-seed Flow-GRPO tables to the revision.
R2 (W1): Size of gains (70.8% vs. 71.1% on Gemini-3-Flash)70.8% vs. 71.1% compares different modes (zero-shot VLMs already have an 8.7-pt pairwise–pointwise gap). Holding mode fixed, our rubrics add +9.0 pts pointwise (62.1% →\rightarrow 71.1%), and switching to pairwise lifts accuracy further to 77.2% (+6.4 over pairwise judge).Add Gemini-3-Flash and Qwen3-VL-8B pairwise results and clarify cross-mode vs. fixed-mode comparisons.
R2 (W2): Reward inference cost during RLWith vLLM prefix caching, 20 rubrics share one image/prompt encoding, matching single-rubric latency (74.2 vs. 74.8 ms/img) and adding ~12% per iteration over HPSv3 (~33 min/run; top-4 rubrics match HPSv3 compute), while using 512 labeled pairs total (0.04–0.08% of scalar reward data).Add vLLM prefix-caching latency and iteration-time benchmarks (Table) and compute-annotation analysis.
R2 (Q1) & R3 (W2/Q1): Validation protocol, RL checkpoints & human studyAll results use cross-corpus validation (256 train + 256 val = 512 pairs; zero MMRB2 overlap) and the fixed step-500 checkpoint (85–95% of 800-step max reward before reward hacking). Prompt-level bootstrap 95% CI for human preference is [39.6%, 46.7%].State the 256+256 budget and step-500 rule in the main text/Appendix B; include new RL reward curves in the main text; fix Appendices G & H; add human-study sampling rule and CI.
R3 (W1/Q2/Q3): Auto-Rubric adaptation, RubricRL & ARR [1]Table 2 uses Auto-Rubric's native OpenJudge pointwise mode; pairwise re-evaluation still shows +3.3–8.1 pt gains across all 3 benchmarks. Pointwise scoring avoids O(K2)O(K^2) calls in K=24K=24 GRPO. Reconciled RubricRL offline caching and contrasted ARR's unweighted prompt merge.Document OpenJudge setup (Appendix D), fix RubricRL description (Appendix C), and cite/discuss ARR [1] in Section 2 and Appendix C.

[1] Juanxi Tian et al. Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria. arXiv:2605.08354v1, 2026.

Dear Reviewers,

We sincerely thank you for your time and effort in reviewing our submission and writing your acknowledgement.

With an hour remaining before the discussion deadline, we would greatly appreciate it if you could carefully review our latest responses and provide further clarifications at your convenience. We highly value any opportunities to address any potential remaining issues, which we believe will be helpful for a higher rating. Your timely feedback is extremely important to us! Once again, thanks so much for your time and effort; we deeply appreciate it!

Best, Authors of Submission

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.