acceptodds
Under review as a conference paper at ICLR 2027

Think Before You Score: Thinking Reward Model for Visual Generation

Abstract

Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and candidate outputs directly to scalar rewards, leaving implicit what should be evaluated for each individual case. We introduce Think Before You Score, a paradigm that explicitly determines what matters for each case before judging how well the candidate performs. Following this principle, we propose the Thinking Reward Model (TRM), which formulates case-adaptive rubrics, performs rubric-guided assessment, and produces fine-grained pointwise rewards. We further observe that conventional pairwise preference optimization can induce score polarization, and introduce Pairwise Dual-Group Relative Policy Optimization (PD-GRPO), which leverages pairwise supervision to improve reward discrimination while preserving fine-grained pointwise scoring. Extensive experiments on image generation and editing reward-modeling benchmarks demonstrate that TRM achieves state-of-the-art performance among open-source reward models while remaining highly competitive with proprietary alternatives. Moreover, using TRM as a reward for reinforcement learning consistently improves diverse visual generation models, demonstrating that its fine-grained, case-adaptive rewards translate into effective optimization signals for visual generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Reward Model, Visual Generation, Image Generation, Image Editing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BcjLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TRM is a generative pointwise reward model for image generation and editing that first writes a case-specific Yes/No checklist, checks each item, and then outputs a 0–10 score. It is trained by SFT on about 48K Gemini-annotated traces, followed by PD-GRPO with pairwise preferences, and is used as the reward for RL algorithms likFlow-GRPO.

Critical strengths
  • The checklist paradigm is reasonable and general. Deciding what to check for each case before scoring is a sensible way to make a reward model's judgment explicit, and the same procedure transfers naturally across tasks (generation and editing here, and potentially video or other modalities).

  • Broad coverage. One reward model handles both generation and editing, and it is used for Flow-GRPO on eight generators, including the already strong SenseNova-U1.5.

Critical weaknesses
  • Train-test overlap. TRM's training data includes Edit-Compass, RISE-Bench and EvalMuse. But EditReward-Compass is built from Edit-Compass, and MMRB2 samples its prompts from RISE-Bench and EvalMuse, while both EditReward-Compass and MMRB2 are used for evaluation.

  • Marginal gain from PD-GRPO. PD-GRPO adds only 1.1 / 2.1 points over SFT (Table 2), and 0.7 / 1.1 under the tie-aware protocol (Table 10).

  • Checklist quality is never evaluated. The checklist is the core of the paper, but its coverage, redundancy and the accuracy of the per-item yes/no judgments are never checked against Gemini or humans.

Questions for authors
  • Which splits of Edit-Compass, RISE-Bench and EvalMuse are used for training? Can you de-duplicate the training data against EditReward-Compass and MMRB2 and re-report Tables 2 and 3?

  • How good are the generated checklists? Can you report their coverage and redundancy against Gemini or human-written checklists, and the agreement between the per-item yes/no judgments and human labels?

  • Can you show concrete cases that SFT ranks wrongly and PD-GRPO fixes, and report multiple runs to confirm the gain?

  • With the same prompt and seed on the eval set, which checklist items get better or worse after RL, and what is the trend?

  • TRM samples at temperature 0.7 during PD-GRPO. How much do the checklist and score vary across samples for the same image, and does PD-GRPO reduce this variance? Which temperature is used when TRM serves as the reward in Flow-GRPO?

9 comments
AuthorsOpenReview ↗

Think Before You Score: Thinking Reward Model for Visual Generation

Response to Weakness 1 and Question 1: Thank you for your question. Below, we explain our data construction rationale, discuss similar practices in prior work, and present downstream RL results to validate TRM's effectiveness in practice

  • We select source images and editing instructions from public benchmarks because they cover representative editing tasks and provide source-image–instruction pairs with clear editing objectives. For each task category, we refer to model rankings on relevant benchmarks to select editing models with different capability levels and generate candidate outputs for each fixed source-image–instruction pair. These candidates cover different quality levels and failure modes, providing fine-grained supervision for TRM's error detection and scoring. To further validate the effectiveness of our method, we conducted a training-data de-duplication experiment. Specifically, we checked the SFT training data against EditReward-Compass and MMRB2. We removed training examples that shared both the source image and editing instruction with evaluation examples, even when their edited images differed. We then retrained TRM(SFT) from the same base model, keeping all other training and evaluation settings unchanged. The results are reported below.

    ModelEditScore-ERB (IF)EditScore-ERB (VC)EditScore-ERB (O)MMRB2EditReward-Compass (IA)EditReward-Compass (VC)
    TRM(SFT)0.7500.6390.74353.00.6260.622
    TRM(SFT, de-duplicated)0.7540.6370.75053.70.6140.607

    After de-duplication, performance on EditScore-ERB remains close to the original results, with absolute changes of at most 0.007 across IF, VC, and Overall. On EditReward-Compass, IA and VC decrease modestly by 0.012 and 0.015, respectively. These limited changes suggest that TRM(SFT)'s performance is not primarily dependent on the identified training–evaluation overlaps. Our method explicitly trains the model to formulate case-adaptive evaluation criteria and assess each candidate against them, providing a structured evaluation procedure that can be applied across different editing instructions and visual content. The largely preserved performance after de-duplication supports the effectiveness of this rubric-guided framework in learning transferable evaluation capabilities. Together with the downstream reinforcement learning improvements in Tables 4 and 5, these findings provide complementary evidence that TRM generalizes as an evaluator and produces useful reward signals for optimizing image generation and editing models.

  • Similar data construction strategies have been used in reward modeling. EditReward collects source images and editing instructions from benchmarks such as GEdit-Bench and ImgEdit-Bench and generates candidates using six editing models [1]. RationalRewards and CheckDistill also use EditReward-Data to train their image editing reward models [2, 3]. ADIEE uses source images from the Emu-Edit test set, creates new global editing and style editing instructions for these images, and generates candidates to train an editing scorer [4]. In language and reasoning tasks, UltraFeedback and C2RM collect tasks from benchmarks such as TruthfulQA, SciEval, and LogiQA, generate candidate responses using multiple models, and construct reward supervision from these responses [5, 6].

  • To further validate TRM's effectiveness in practice, we evaluate its effectiveness as a reward model for reinforcement learning across three base image generation models and five image editing model configurations. As shown in Tables 4 and 5, TRM-guided reinforcement learning improves FLUX.1-dev's TIIF-Long score from 74.82 to 81.38 and BAGEL's ImgEdit score from 3.37 to 3.91. BAGEL's overall scores on GEdit-Bench-EN and GEdit-Bench-CN also improve from 6.95 and 6.98 to 7.57 and 7.64, respectively.These results further demonstrate that TRM provides effective reward signals for different image generation and editing models, improving downstream performance.

References

[1] Wu et al. EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing. ICLR 2026.

[2] Wang et al. RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time. COLM 2026.

[3] Cai et al. CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing. arXiv:2610.26740, 2026.

[4] Chen et al. ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation. ICCV 2025.

[5] Cui et al. UltraFeedback: Boosting Language Models with Scaled AI Feedback. ICML 2024.

[6] He et al. Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning. EMNLP 2025.

Response to Weakness 2: Thank you for your suggestion. PD-GRPO’s gains are achieved on top of the strong TRM(SFT) baseline. TRM(SFT) already achieves state-of-the-art performance among open-source models on multiple reward-model benchmarks, and PD-GRPO further improves its preference prediction capability, with more pronounced gains on image editing benchmarks. Below, we explain the performance of the SFT baseline, the optimization objective of PD-GRPO, and its empirical gains.

  • We use high-quality evaluation data to provide direct supervision for checklist generation, criterion-level judgments, and final score prediction, enabling the general-purpose backbone to learn the full Think Before You Score reward-modeling paradigm through structured SFT. As shown in Tables 2 and 3, TRM(SFT) achieves state-of-the-art performance among open-source models on multiple reward-model benchmarks for text-to-image generation and image editing, providing a strong starting point for subsequent PD-GRPO optimization.
  • PD-GRPO introduces pairwise preference supervision to help independently assigned image scores reflect subtle quality differences between candidates. For example, two edited images may both complete the requested modification, while one blends more naturally with the original scene. Pairwise preferences provide a direct training signal for such differences.
  • Starting from the strong TRM(SFT) baseline, PD-GRPO achieves further gains in reward-model evaluation for both text-to-image generation and image editing. In Table 2, TRM(SFT) already achieves accuracies of 70.1% and 65.8% on the two reward-model benchmarks for image generation, and PD-GRPO further improves these results by 1.1 and 2.1 percentage points, respectively. The gains are more pronounced for image editing: in Table 3, preference prediction accuracy improves by 5.2 and 3.5 percentage points on MMRB2 Edit and EditReward-ERB 2-path, respectively.

Response to Question 2: Thank you for your suggestion. To assess checklist quality, we randomly sampled 100 examples and evaluated TRM-generated checklists against human annotations in terms of coverage, redundancy, and agreement on criterion-level judgments. The checklists achieved 95% coverage, with redundancy found in 3 of the 100 checklists. Agreement between TRM’s criterion-level judgments and human annotations reached 94.82%. During data construction, human experts established and refined the annotation procedure, while a verifier based on Qwen3.5-397B-A17B further identified missing requirements and removed redundant or ambiguously worded checklist items to improve the quality of structured supervision.

Response to Question 3: Thank you for your suggestion. As shown in the case figures, TRM(SFT) produces rankings inconsistent with human preferences, whereas TRM(RL) produces rankings aligned with human preferences after PD-GRPO training. These cases illustrate how PD-GRPO corrects SFT’s preference prediction errors. Across all evaluation benchmarks, we evaluate both models using identical images and prompts, with the decoding temperature uniformly set to 0 to ensure stable and consistent evaluation.

Response to Question 4: Thank you for your question. We first analyze the improvements in checklist quality and criterion-level judgment accuracy following PD-GRPO training, and then examine how these improvements are reflected in reward-model benchmark results across the corresponding evaluation dimensions.

  • During evaluation, we compare TRM(SFT) and TRM(RL) using identical inputs and set the decoding temperature to 0 for both models to maintain evaluation stability. After PD-GRPO training, the generated checklist items become more specific and easier to verify, and the criterion-level judgments become more accurate. For example, in image editing, the checklist explicitly checks whether the target object's specified attributes have been modified as requested and whether content outside the edited region has been preserved.
  • These improvements to the checklist items are also reflected in higher scores on corresponding evaluation dimensions, such as instruction following and source consistency. In Table 2, TRM's preference prediction accuracy improves on both GenAI-T2I and MMRB2-T2I. In Table 3, instruction following, visual consistency, and overall evaluation on EditScore-ERB all improve, with instruction following increasing from 0.750 to 0.786 and visual consistency from 0.639 to 0.674. IA and VC on EditReward-Compass, as well as preference prediction accuracy on MMRB2 Edit and EditReward-ERB 2-path, also improve.

Response to Question 5: Thank you for your question. Below, we report score variability during PD-GRPO training and clarify the decoding temperature used for downstream reward evaluation.

  • Score variability for the same image and the effect of PD-GRPO. We compare score variability at the beginning and final step of PD-GRPO training using repeated samples for the same image. The average within-group variance of normalized scores decreases from 0.22 to 0.09, with an overall downward trend during training, indicating more stable scores for the same image after training. From an algorithmic perspective, GRPO updates the policy based on within-group relative advantages, increasing the probability of sampling outputs with higher rewards. This encourages more concentrated scores, consistent with the observed decrease in variance.

  • Decoding temperature in downstream Flow-GRPO. In downstream Flow-GRPO, we set TRM's decoding temperature to 0 to improve the stability of reward scores.

Reviewer_2hPjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Thinking Reward Model (TRM), which generates case-adaptive evaluation rubrics before assessing an image and producing a pointwise score. TRM is trained through structured supervised fine-tuning followed by pairwise preference optimization using PD-GRPO. Experiments evaluate both preference prediction and downstream reinforcement learning across image generation and editing models.

Critical strengths
  • The case-adaptive rubric design provides an explicit evaluation process while retaining a pointwise scoring interface suitable for downstream reinforcement learning.
  • The evaluation covers multiple reward-modeling benchmarks and generation and editing backbones. The downstream improvements provide evidence of practical utility beyond preference-prediction accuracy alone.
Critical weaknesses

The specific contribution of the “Think Before You Score” design is not sufficiently isolated. Although the paper compares TRM with direct-scoring reward models such as HPSv3, differences in backbones, training data, and optimization make it difficult to attribute the gains to explicit reasoning or case-adaptive rubrics. A controlled comparison with score-only and reasoning-without-rubric variants would better establish the value of the proposed evaluation process.

Questions for authors

With the same backbone, training datas, and comparable training budgets, how does TRM compare with variants that directly predict scores or generate reasoning without explicit rubrics, in terms of preference accuracy and downstream optimization?

3 comments
AuthorsOpenReview ↗

Think Before You Score: Thinking Reward Model for Visual Generation

Response to Weakness 1 and Question 1:

  • Thank you for your question. To assess the contribution of the “Think Before You Score” evaluation process itself, we use Qwen3.5-9B as the common backbone and compare three variants using the same amount of training data and the same training resources: direct scoring, reasoning-based scoring without explicit rubrics, and TRM with case-adaptive rubrics and criterion-level verification. We evaluate these variants through reward-model benchmarks and controlled downstream reinforcement learning experiments to examine whether the proposed process improves evaluation capability and whether this improvement translates into more effective optimization signals.

    • Reward Model Benchmark (Preference Accuracy): We first compare the reward-modeling performance of the three variants on EditScore-ERB and EditReward-ERB to assess how different scoring paradigms affect preference prediction. The results are as follows:

      ModelEditScore-ERB (IF)EditScore-ERB (VC)EditScore-ERB (O)EditReward-ERB 2-path
      Qwen-3.50.4400.3080.40133.8
      Direct Scoring0.7230.5780.67856.3
      Reasoning0.6740.5800.65055.6
      TRM0.7500.6390.74367.8

      On EditScore-ERB, TRM achieves 0.750, 0.639, and 0.743 for instruction following, visual consistency, and overall evaluation, respectively. Its overall score exceeds those of direct scoring and reasoning-based scoring without explicit rubrics by 0.065 and 0.093, respectively. On EditReward-ERB, TRM achieves a preference prediction accuracy of 67.8%, outperforming the two variants by 11.5 and 12.2 percentage points, respectively.

    • Downstream Optimization: To further compare the effects of different evaluation paradigms on downstream optimization, we use BAGEL as the base model and perform reinforcement learning with direct scoring, reasoning-based scoring without explicit rubrics, and TRM as reward models, keeping all other training settings unchanged. We compare the three optimized models on ImgEdit and GEdit-Bench (EN/CN). The results are as follows:

      ModelImgEdit OverallGEdit-EN G_SCGEdit-EN G_PQGEdit-EN G_OGEdit-CN G_SCGEdit-CN G_PQGEdit-CN G_O
      BAGEL3.377.896.686.957.926.766.98
      + RL (Direct Scoring)3.738.166.897.278.126.867.23
      + RL (Reasoning)3.698.266.897.338.226.977.36
      + RL (TRM)3.918.397.267.578.407.337.64

      TRM-guided RL outperforms both control variants on all metrics. On ImgEdit, TRM achieves an overall score of 3.91, exceeding direct scoring and reasoning without explicit rubrics by 0.18 and 0.22, respectively. It also achieves the highest overall scores on GEdit-Bench-EN and CN, reaching 7.57 and 7.64. Under the same training settings, TRM achieves better downstream performance, indicating that an evaluation paradigm based on adaptive rubrics and criterion-level verification is more effective.

As the discussion period is coming to a close, we wanted to follow up briefly on our rebuttal. We would be grateful to know whether our responses resolved your concerns. If any points remain unclear, or if further experiments or clarifications would be helpful, we would be glad to provide them in the time remaining. Thank you again for your time and for your thoughtful feedback on our paper.

Thank the author for the efforts to address my concern. I have no further concern and I would keep my positive rating.

Reviewer_p4enOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes TRM, a visual reward model that generates case-specific evaluation rubrics, assesses the candidate against them, and produces a pointwise score. Training combines supervised learning on approximately 48K structured examples with PD-GRPO on approximately 4K preference pairs. PD-GRPO aims to improve preference discrimination without continually expanding score gaps. The evaluation covers reward-model benchmarks for image generation and editing, as well as downstream reinforcement learning across several generative models.

Critical strengths
  1. The evaluation goes beyond preference prediction and tests whether TRM provides useful optimization signals. Improvements across several generation and editing models are meaningful evidence of practical utility.

  2. Appendix D.3 reports predicted tie rates and tie-aware accuracy, and the paper evaluates the original Qwen model under the same pointwise interface. These details help explain how the evaluation protocol affects the results.

Critical weaknesses
  1. Excluding predicted ties in the main table makes the cross-model accuracy comparisons difficult to interpret. TRM(RL) scores 71.2% / 67.9% in Table 2, but 68.4% / 63.9% when ties receive half credit in Appendix Table 10. This is a material change in the apparent margin over other methods. The appendix provides the latter results for TRM and its initialization, but not for all major baselines under one common protocol. The claim of outperforming all evaluated open-source reward models therefore needs to be checked under consistent treatment of ties and parsing failures.

  2. The experiments do not yet isolate the contributions of adaptive rubrics and PD-GRPO. SFT simultaneously introduces substantial new data, teacher annotations, longer structured reasoning, and adaptive rubrics. There is no matched comparison with direct scoring or fixed rubrics using the same data and training budget. For PD-GRPO, Figure 7 shows a less polarized score distribution, but a more spread-out distribution is not by itself evidence of better calibration or better RL guidance. A controlled comparison with the BT formulation on preference accuracy, pointwise score quality, and downstream optimization would make the argument stronger.

Questions for authors
  1. Could all major baselines in Table 2 be evaluated with the same all-pair protocol, assigning half credit to ties and reporting parsing success rates?
  2. With the same annotation data and training budget, how do direct scoring, fixed rubrics, and adaptive rubrics compare?
  3. Under matched training conditions, how do PD-GRPO and the BT formulation compare on preference accuracy, error against independent pointwise annotations, and downstream RL performance?
  4. Appendix A uses sources such as Edit-Compass for training, while related benchmarks appear in evaluation. How are training and external evaluation data separated at the source-image, instruction, and candidate-output levels? The stated prompt-level training/validation split does not by itself establish this separation.
4 comments
AuthorsOpenReview ↗

Think Before You Score: Thinking Reward Model for Visual Generation

Response to Weakness 1 and Question 1:

Thank you for your suggestion. To further validate TRM's preference prediction capability, we conduct additional evaluations of TRM and other models on GenAI-T2I and MMRB2-T2I using the same tie-handling rule. The results are as follows:

ModelGenAI-T2IMMRB2-T2I
Qwen2.5-VL-7B-Instruct54.251.7
Qwen3-VL-8B-Instruct57.755.1
Qwen3.5-9B54.653.5
UnifiedReward62.357.4
RationalRewards66.961.9
TRM(SFT)67.762.8
TRM(RL)68.463.9

Under this protocol, TRM(RL) maintains strong performance, achieving accuracies of 68.4% on GenAI-T2I and 63.9% on MMRB2-T2I. It outperforms RationalRewards, the strongest competing reward model in the table, by 1.5 and 2.0 percentage points, respectively. The corresponding accuracy gains over its Qwen3.5-9B backbone are 13.8 and 10.4 percentage points.

Response to Weakness 2 and Question 2:

Thank you for your suggestion. To examine the contribution of adaptive rubrics, we use Qwen3.5-9B as the common backbone and compare direct scoring, scoring with fixed rubrics, and TRM with adaptive rubrics, using the same amount of training data and the same training budget. We evaluate the three variants on EditScore-ERB and EditReward-ERB. The results are as follows:

ModelEditScore-ERB (IF)EditScore-ERB (VC)EditScore-ERB (O)EditReward-ERB 2-path
Qwen-3.50.4400.3080.40133.8
Direct Scoring0.7230.5780.67856.3
Fixed Rubrics0.6740.5800.65055.6
TRM0.7500.6390.74367.8

On EditScore-ERB, TRM achieves 0.750, 0.639, and 0.743 for instruction following, visual consistency, and overall evaluation, respectively. Its overall score exceeds those of direct scoring and scoring with fixed rubrics by 0.065 and 0.093, respectively. On EditReward-ERB, TRM achieves a preference prediction accuracy of 67.8%, outperforming the two control variants by 11.5 and 12.2 percentage points, respectively. These results indicate that an evaluation paradigm based on adaptive rubrics and criterion-level verification is more effective.

Response to Weakness 2 and Question 3:

Thank you for your suggestion. To compare the effectiveness of PD-GRPO and BT, we evaluate them on reward-model benchmarks and in downstream reinforcement learning.

  • Reward-Model Benchmark Comparison: We compare the reward models optimized with PD-GRPO and BT on EditScore-ERB and EditReward-ERB. The results are as follows:

    ModelEditScore-ERB (IF)EditScore-ERB (VC)EditScore-ERB (O)EditReward-ERB 2-path
    Qwen-3.50.4400.3080.40133.8
    TRM(SFT)0.7500.6390.74367.8
    BT0.6000.5460.56554.9
    TRM(RL)0.7860.6740.77371.3

    After PD-GRPO optimization, TRM achieves 0.786, 0.674, and 0.773 for instruction following, visual consistency, and overall evaluation on EditScore-ERB, respectively, with the overall score exceeding BT by 0.208. On EditReward-ERB, TRM achieves a preference prediction accuracy of 71.3%, outperforming BT by 16.4 percentage points.

  • Downstream RL Comparison: We use the BT-optimized reward model and TRM to guide separate downstream reinforcement learning runs and compare the resulting models on ImgEdit and GEdit-Bench (EN/CN). The results are as follows:

    Reward ModelImgEdit OverallGEdit-EN G_SCGEdit-EN G_PQGEdit-EN G_OGEdit-CN G_SCGEdit-CN G_PQGEdit-CN G_O
    BT3.588.066.677.088.006.727.10
    TRM3.918.397.267.578.407.337.64

    Using TRM as the reward model yields an ImgEdit Overall score of 3.91, exceeding that obtained with the BT-optimized reward model by 0.33. On GEdit-Bench-EN and CN, the overall scores reach 7.57 and 7.64, exceeding the corresponding BT results by 0.49 and 0.54, respectively.

AuthorsOpenReview ↗

Think Before You Score: Thinking Reward Model for Visual Generation

Response to Question 4:

Thank you for your question. Thank you for your question. Below, we explain our data construction rationale, discuss similar practices in prior work, and present downstream RL results to validate TRM's effectiveness in practice

  • We select source images and editing instructions from public benchmarks because they cover representative editing tasks and provide source-image–instruction pairs with clear editing objectives. For each task category, we refer to model rankings on relevant benchmarks to select editing models with different capability levels and generate candidate outputs for each fixed source-image–instruction pair. These candidates cover different quality levels and failure modes, providing fine-grained supervision for TRM's error detection and scoring.

  • Similar data construction strategies have been used in reward modeling. EditReward collects source images and editing instructions from benchmarks such as GEdit-Bench and ImgEdit-Bench and generates candidates using six editing models [1]. RationalRewards and CheckDistill also use EditReward-Data to train their image editing reward models [2, 3]. ADIEE uses source images from the Emu-Edit test set, creates new global editing and style editing instructions for these images, and generates candidates to train an editing scorer [4]. In language and reasoning tasks, UltraFeedback and C2RM collect tasks from benchmarks such as TruthfulQA, SciEval, and LogiQA, generate candidate responses using multiple models, and construct reward supervision from these responses [5, 6].

  • To further assess TRM's performance, we evaluate its effectiveness as a reward model for reinforcement learning across three base image generation models and five image editing model configurations. As shown in Tables 4 and 5, TRM-guided reinforcement learning improves FLUX.1-dev's TIIF-Long score from 74.82 to 81.38 and BAGEL's ImgEdit score from 3.37 to 3.91. BAGEL's overall scores on GEdit-Bench-EN and GEdit-Bench-CN also improve from 6.95 and 6.98 to 7.57 and 7.64, respectively. These results further demonstrate that TRM provides effective reward signals for different image generation and editing models, improving downstream performance.

References

[1] Wu et al. EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing. ICLR 2026.

[2] Wang et al. RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time. COLM 2026.

[3] Cai et al. CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing. arXiv preprint arXiv:2610.26740, 2026.

[4] Chen et al. ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation. ICCV 2025.

[5] Cui et al. UltraFeedback: Boosting Language Models with Scaled AI Feedback. ICML 2024.

[6] He et al. Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning. EMNLP 2025.

As the discussion period is coming to a close, we wanted to follow up briefly on our rebuttal. We would be grateful to know whether our responses resolved your concerns. If any points remain unclear, or if further experiments or clarifications would be helpful, we would be glad to provide them in the time remaining. Thank you again for your time and for your thoughtful feedback on our paper.

Dear Reviewer p4en,

Thank you again for your time and thoughtful review. We would appreciate it if you could review our responses when you have a chance. If any questions or concerns remain, we would be happy to provide further clarification. We will continue to follow the discussion and respond promptly until the rebuttal deadline.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.