acceptodds
Under review as a conference paper at ICLR 2027

CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing

Abstract

Reinforcement learning for instruction-guided image editing requires reward models that provide reliable, accurate, and efficient feedback. Direct scalar prediction compresses multidimensional editing preferences, whereas generative evaluators can expose their judgment through explicit reasoning. However, broad scoring criteria do not specify all instance-dependent editing and preservation constraints, and generating long reasoning trajectories is costly for online optimization. We introduce CheckDistill, a pointwise reward modeling framework that internalizes instruction-adaptive checklists. We decompose each editing condition into atomic, independently verifiable requirements and construct four-dimensional scoring labels together with structured verification evidence. After supervised score initialization, a privileged teacher conditions on checklists, item-level judgments, and dimension-level rationales to supervise the student's own scoring trajectories. A reliability-weighted on-policy self-distillation objective adjusts supervision at score tokens according to agreement between the teacher's expected scores and reference labels, while retaining candidate probability weights in a truncated token support. The deployed student directly predicts four score distributions without checklist inputs or explicit reasoning and aggregates their expectations into a scalar reward. On Qwen3.5-9B, CheckDistill achieves aggregate accuracies of 72.30%, 56.67% and 82.90% on MMRB2, MER-Bench and EditReward-Bench, respectively, exceeding the compared open-source baselines on all three metrics.

Primary area:
generative models
Keywords:
Image Editing, Reward Model, On-Policy Self-Distillation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_wcJ8OpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses instance-specific checklists to represent fine-grained constraints in image editing tasks. It then proposes CR-OPSD to distill checklist- and reasoning-based teacher supervision into a model that directly predicts four-dimensional reward scores. The main goal is to preserve detailed evaluation capabilities while reducing the inference cost of long reasoning traces during online RL.

Critical strengths

The paper has a clear problem formulation and presents a coherent pipeline covering checklist construction, privileged distillation, and downstream RL. The method achieves strong results across several reward benchmarks while substantially reducing the number of output tokens required during inference. This makes the approach practically relevant for large-scale reward evaluation.

Critical weaknesses
  1. The central claim is that reasoning can be effectively distilled away, but the contribution of the original reasoning is not sufficiently analyzed. The current experiments mainly compare final accuracy and output length. They do not fully address: how much reasoning contributes to different types of examples; first, whether the distilled direct scorer degrades on long-tail cases; then, whether the reasoning and distilled models have comparable calibration; and whether the distilled model is more vulnerable to distribution shift. The authors should construct additional OOD samples, including small localized edits, multi-object edits, negation, complex spatial relations, unseen attribute combinations, outputs from unseen editing models, and different image-quality distributions. The reasoning model, direct SFT model, and CheckDistill should then be compared.

  2. The training labels and checklists rely heavily on Gemini-generated supervision. Although human preferences are used for filtering, the final dimension scores and rationales are still mainly generated by an automatic teacher. The paper should report agreement between automatic annotations and human judgments, or evaluate robustness across multiple independent teachers.

  3. The efficiency analysis focuses mainly on token count and does not fully measure deployment cost.

Questions for authors

See Critical Weaknesses

4 comments

Thank you for your careful reading of our submission and for the clear suggestions on how to strengthen the evaluation. We respond to each of the three points below.


W1: OOD Evaluation and Analysis

Following your suggestion, we construct an OOD benchmark for evaluation in Appendix J. We randomly sample 51 tasks from CPI-Intelligent-Bench [1], covering fine-grained local edits, multi-object edits, edits with multiple reference images, negated instructions, complex spatial relations, and unseen attribute combinations. For each task, human annotators rank three candidate edits generated by Nano Banana Pro, Qwen-Image-2.0-Pro, and Seedream 5.0 Pro without knowing which model produced each image. Figures 11 and 12 provide visualizations of the benchmark.

On the OOD benchmark, we use 3-pair ranking accuracy to compare eight model variants, including Gemini-3.0-Flash with the data construction template (Appendix D) and with the scoring template (Appendix E), Qwen3.5-9B (None → S) and three SFT variants (None → S, None → R+S, and None → C+J+S) from Section 4.3, and the reasoning and distilled variants of CheckDistill.

As shown in Table, distilled CheckDistill achieves strong performance on the OOD benchmark, attaining 25.49% ranking accuracy and outperforming direct-scoring SFT (17.65%) and SFT with explicit rationales (21.57%). Its performance is comparable to both SFT (None →\to C+J+S) and the reasoning variant of CheckDistill, highlighting the benefits of checklist-guided reasoning for tasks requiring fine-grained visual understanding. We further present a scoring example for the reasoning and distilled variants of CheckDistill (Figures 13 and 14). The two variants show similar scoring behavior and have comparable calibration. These results demonstrate that CheckDistill effectively internalizes the evaluation capabilities of the reasoning model without requiring explicit reasoning at inference time.

ModelAccuracy ↑
Gemini-3.0-Flash (data construction template)29.41%
Gemini-3.0-Flash (scoring template)23.53%
Qwen3.5-9B (None → S)3.92%
SFT (None → S)17.65%
SFT (None → R+S)21.57%
SFT (None → C+J+S)25.49%
CheckDistill (Reasoning)25.49%
CheckDistill (Distilled)25.49%

Table: Ranking accuracy on the OOD benchmark.

References:

  • [1] Zhou et al., CPI-Bench: A Comprehensive, Practical and Intelligent Benchmark for Real-World Image Editing, arXiv 2026.

W2: Agreement between automatic annotations and human judgments

We add a human evaluation of the automatic annotations in CheckDistill-40K and include the analysis in Appendix K. We randomly sample 100 examples and assess five dimensions of annotation quality: score agreement, checklist correctness, checklist completeness, reasoning correctness, and score-reason consistency.

For score agreement, we compute the mean absolute score deviation (Δ\Delta) to measure how closely the automatically assigned scores (sevaluators_{\text{evaluator}}) match the human ratings (shumans_{\text{human}}). We define (Δ\Delta) as the average of (∣sevaluator−shuman∣\lvert s_{\text{evaluator}} - s_{\text{human}}\rvert) across examples.

For the remaining four dimensions, human annotators rate the automatic annotations on a scale from 1 to 5. We extend this assessment to 2K examples using GPT-6.1 Sol with the consistency assessment template in Appendix K.

The score annotations in CheckDistill-40K agree well with human judgments (Δ=0.99\Delta = 0.99). Both human annotators and GPT-6.1 Sol give favorable ratings for checklist correctness, checklist completeness, reasoning correctness, and score-reason consistency. In addition, for GPT-6.1 Sol, the difference between the data construction template (Δ=0.45\Delta = 0.45) and the scoring template (Δ=0.65\Delta = 0.65) indicates that checklist-based data construction yields score annotations that agree more closely with human ratings.

EvaluatorIF (Δ\Delta, ↓)AP (Δ\Delta, ↓)CP (Δ\Delta, ↓)PQ (Δ\Delta, ↓)Overall (Δ\Delta, ↓)
Gemini-3.0-Flash1.550.700.552.700.99
GPT-6.1 Sol (data construction template)0.900.450.100.800.45
GPT-6.1 Sol (scoring template)1.350.600.350.800.65

Table: Evaluator score agreement with human ratings. Δ\Delta is computed as the average of ∣sevaluator−shuman∣\lvert s_{\text{evaluator}} - s_{\text{human}}\rvert across examples. Lower Δ\Delta indicates closer agreement.

EvaluatorChecklist Correctness ↑Checklist Completeness ↑Reasoning Correctness ↑Score-Reason Consistency ↑
GPT-6.1 Sol4.784.583.494.67
Human4.864.664.034.82

Table: Checklist and Rationale Consistency of Automatic Annotations as Assessed by GPT-6.1 Sol and Human Annotators.

W3: Efficiency Analysis

We extend our inference efficiency evaluation to include mean generation time per request, throughput, and total runtime alongside token counts and accuracy. We conduct these experiments on MMRB2 Single and include this analysis in Appendix L as well.

CheckDistill processes 1.62× as many pairs per second as SpatialReward and 2.57× as many as EditScore, while achieving the highest accuracy among the three models.

ModelInput tokensOutput tokensGeneration time (s/request) ↓Throughput (pairs/s) ↑Total runtime ↓Accuracy ↑
EditScore8,6393464.910.19570m 26s57.90%
SpatialReward7,1024962.980.31044m 09s67.10%
CheckDistill4,617260.760.50127m 20s73.36%

Table: Efficiency and accuracy on MMRB2 Single.

Thank you again for your thoughtful feedback. We hope our responses address your concerns and would be happy to discuss any remaining questions.

Reviewer_wcJ8OpenReview ↗

Review

Thank you for the response. Follow-up on Weakness 1: (1) Beyond a single qualitative example (Figs. 13–14), could you report quantitative calibration or score agreement (e.g., MAE, correlation) between the reasoning and distilled variants across the benchmark? Besdes, since 51 OOD tasks is relatively small, could you provide a breakdown by edit type (e.g., negation, spatial relations, local edits) to check whether distillation degrades on specific long-tail cases?

We are glad to receive your follow-up comments! We report score agreement between the reasoning and distilled variants across the OOD benchmark in Table 12 of Appendix J, with an overall mean absolute score deviation of 0.85, indicating close agreement.

We further provide accuracy by edit type on the OOD benchmark in the table below. For CheckDistill-40K, negated instructions (1.5% of the dataset) and attribute combinations (including unseen attributes in the OOD benchmark) represent long-tail cases, while each of the other types accounts for over 10% of CheckDistill-40K. As you anticipated, the distilled model shows some decline in accuracy compared with the reasoning model on these long-tail cases. Nevertheless, compared with the direct SFT model, CheckDistill achieves a better balance between accuracy and efficiency.

We believe the decline may stem from limited training coverage of these long-tail cases, making it harder for CheckDistill to generalize its implicit evaluation capabilities to them. This gap could be reduced by expanding the reward training data to include more long-tail data. Alternatively, it could be mitigated during RL by using CheckDistill as an efficient reward model for most editing tasks and switching to a reasoning model for challenging OOD cases.

ModelSmall Local EditNegated InstructionComplex Spatial RelationMulti-Object EditAttribute CombinationOverall Accuracy ↑
SFT (None → S)14.29%16.67%30.00%25.00%7.14%17.65%
CheckDistill (Reasoning)14.29%33.33%30.00%25.00%21.43%25.49%
CheckDistill (Distilled)28.57%25.00%40.00%37.50%7.14%25.49%

Table: Ranking accuracy on the OOD benchmark (including accuracy by edit type).

We sincerely appreciate your time. We hope this response addresses your concerns and wish you all the best.

Reviewer_tg3vOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces CheckDistill, a pointwise reward model for instruction-guided image editing that internalizes fine-grained, instance-specific checklist verification into efficient direct scoring. The authors decompose each editing condition into source-grounded atomic checklist items covering instruction following, attribute preservation, and context preservation, while evaluating perceptual quality with a fixed rubric. These checklists, together with item-level judgments, dimension-wise rationales, and four-dimensional score labels, are automatically annotated and filtered according to human preference consistency to construct CheckDistill-40K. Training consists of mixed supervised fine-tuning followed by Checklist-Guided Reliability-Aware On-Policy Self-Distillation, where a privileged teacher observes the checklist-derived evidence and supervises a student that only receives the instruction, source image, and edited result. At inference time, the model directly predicts four score distributions without generating checklists or rationales. Experiments evaluate reward-model accuracy on MMRB2, MER-Bench, and EditReward-Bench, provide component and privileged-information ablations, and apply CheckDistill as the reward model for DiffusionNFT-based online RL of Qwen-Image-Edit on three editing benchmarks.

Critical strengths

This paper has two clear strengths:

  1. CheckDistill decomposes each editing condition into source-grounded atomic checklist items covering instruction following, attribute preservation, and context preservation. This provides more adaptive, traceable, and task-specific evaluation criteria than fixed holistic rubrics.

  2. CheckDistill achivees an effective balance between accuracy and efficiency. During training, a privileged teacher uses checklists, item-level judgments, and dimension-wise rationales for fine-grained evaluation, while reliability-aware on-policy self-distillation internalizes this knowledge into the student. At inference time, the student directly predicts four scores without generating checklists or lengthy reasoning, making it suitable for efficient online RL.

Critical weaknesses

This paper still has two problems:

  1. The online RL experiment lacks sufficiently strong reward-model baselines. Table 2 compares the CheckDistill-guided policy mainly against the base Qwen-Image-Edit and UniWorld-Qwen-Image-Edit. However, UniWorld uses a frozen, training-free Qwen2.5-VL-32B as its reward model, with the reward obtained directly from prompted score-token logits. Although this is a useful general-purpose MLLM baseline, it does not represent recent reward models specifically trained for image-editing evaluation. Therefore, outperforming UniWorld does not establish whether CheckDistill provides a better RL signal than existing specialized editing reward models. A stronger comparison would include a Qwen-Image-Edit policy post-trained using a dedicated reward model, such as FIRM-Qwen-Edit guided by FIRM-Edit-8B, or controlled variants trained with EditScore, SpatialReward, JRM, or Edit-RRM when their implementations are available.
  2. The paper does not sufficiently investigate editing-specific reward hacking. Under the current aggregation, a source-copy result with IF=0, AP=9, CP=9, and PQ=9 still receives a reward of approximately 5.98/9 despite performing no edit. This may encourage the policy to favor source preservation, minimal changes, or partial edits over faithfully executing difficult instructions.
Questions for authors
  1. Can you compare against at least one Qwen-Image-Edit model post-trained with a specialized image-editing reward model, such as FIRM-Qwen-Edit?
  2. How does the proposed reward design prevent the policy from exploiting preservation scores through no-op or under-editing behavior? Can you provide evidence that such behavior does not emerge during RL training?
3 comments

We would like to thank the reviewer for the detailed feedback and concrete suggestions. We address each weakness together with its corresponding question below.


W1&Q1: RL with Specialized Reward Models

The public FIRM-Qwen-Edit adapter is configured for a FLUX transformer, so we post-train Qwen-Image-Edit ourselves using FIRM-Edit-8B as the reward model. Under the same DiffusionNFT setup, we also run RL with two open-source reward models specialized for image editing, EditScore and SpatialReward.

CheckDistill provides a better reward signal for downstream RL than specialized image editing reward models. As shown in the table below, CheckDistill outperforms EditScore, SpatialReward, and FIRM-Edit on all three benchmarks.

ModelImgEdit-Bench↑REDEdit-Bench↑GEdit-EN↑
Qwen-Image-Edit4.294.108.18
EditScore + DiffusionNFT4.354.168.21
SpatialReward + DiffusionNFT4.324.238.26
FIRM-Qwen-Edit*4.42-7.84
FIRM-Edit + DiffusionNFT4.264.138.19
CheckDistill + DiffusionNFT4.464.268.28

Table: Downstream RL results with different reward models. FIRM-Qwen-Edit uses FIRM-Edit to guide RL on Qwen-Image-Edit. * indicates its scores are taken from [1].

W2&Q2: Investigating Editing-Specific Reward Hacking

We conduct an additional experiment in which we randomly sample 200 rollout groups from RL training, each containing eight generated images. We then add the corresponding unedited source image to each group as a baseline. The source copy baselines receive a mean reward of 5.23, while the mean reward across the original rollout groups is 6.46, with an example shown in Figure 10 of Appendix I. This reward gap suggests that CheckDistill provides a signal that encourages the editing model to follow the instruction.

In addition, we add training curves for all four reward dimensions to Figure 9 of Appendix I. The figure shows that the overall reward and all four dimension scores, including instruction following, steadily increase during RL training. Taken together, these results suggest that the policy does not exploit preservation scores through no-ops or under-editing.

Thank you again for providing clear and actionable comments; we hope we were able to address all concerns, and remain available for further discussion.

References:

  • [1] Zhao et al., Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation, arXiv 2026.

Thank you for the response. Your follow-up experiments has addressed my concerns. I have raised my soundness score.

Thank you for engaging with our response and for raising the soundness score. We appreciate your time and support.

Reviewer_tDQqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper presents CheckDistill, an efficient reward modeling framework for instruction-guided image editing. The authors construct CheckDistill-40K, a dataset featuring instance-specific atomic checklists, structured judgments, and four-dimensional quality scores, with human preference-consistency filtering. They further introduce CR-OPSD, which distills checklist-guided evaluation into a direct-scoring reward model through reliability-aware on-policy self-distillation, avoiding explicit reasoning at inference time. Experiments demonstrate competitive performance across three reward benchmarks and consistent improvements in downstream image editing through RL post-training.

Critical strengths

S1. It introduces fine-grained reward supervision. The construction of CheckDistill-40K is a meaningful contribution. Instance-specific atomic checklists provide structured supervision for evaluating editing requirements and preservation constraints, going beyond coarse-grained quality judgments.

S2. The empirical performance is impressive. CheckDistill achieves strong results across three reward benchmarks and multiple model scales. The method substantially reduces output tokens compared with reasoning-based scoring while maintaining high accuracy. Its effectiveness is further supported by improvements on three downstream image-editing benchmarks.

Critical weaknesses

W1. The methodological novelty is limited and attribution of improvements is insufficient. CR-OPSD extends existing OPSD with checklist-derived privileged information and reliability-aware weighting. While the ablations support the effectiveness of these components, the contribution of the proposed on-policy distillation strategy remains insufficiently isolated from additional training and privileged supervision. More controlled comparisons with simpler training alternatives would help establish its incremental benefit.

W2. The downstream comparisons are insufficient. The downstream RL experiments demonstrate improvements over the original editing model, but do not compare different reward models under the same RL setup. Consequently, it remains unclear whether CheckDistill provides superior downstream optimization signals compared with existing reward models.

Questions for authors

Q1. Section 3.2 states that Stage-I SFT uses a 10K subset, while Section 4.3 describes SFT on CheckDistill-40K. Could the authors clarify the dataset size used for the SFT baselines in Figure 5? More importantly, even if the full 40K dataset is used, the comparison does not isolate the contribution of CR-OPSD from additional training or checklist-derived privileged supervision. Could the authors provide a data- and compute-matched continued-SFT baseline or a simpler offline distillation baseline using the same privileged information to better establish the incremental benefit of CR-OPSD?

Q2. Can the authors compare CheckDistill with existing reward models, such as EditScore and SpatialReward, under the same DiffusionNFT training setup? This would clarify whether the improvements in reward benchmark accuracy translate into superior downstream RL performance.

3 comments

Thank you for your thorough assessment of our submission and for suggesting more controlled comparisons. We have grouped the related weaknesses and questions and address them below.


W1&Q1: The Benefit of CR-OPSD

For a fair comparison, all SFT baselines in Figure 5 use the full 40K examples in CheckDistill-40K, matching CheckDistill’s training size. Following your suggestion, we add an offline distillation baseline that starts from the same Stage-I checkpoint. We use the same privileged information as CR-OPSD to generate teacher outputs for offline supervision, with matched training data volume and compute budgets.

CheckDistill improves over offline distillation on MMRB2 (66.35% → 72.30%), MER-Bench (50.80% → 56.67%), and EditReward-Bench (78.45% → 82.90%). These gains, together with the ablation results in Table 3 of the paper, demonstrate the effectiveness of CR-OPSD as a training strategy.

MethodMMRB2 Single ↑MMRB2 Multi ↑MMRB2 Overall ↑MER 2-pair ↑MER 3-pair ↑MER 4-pair ↑MER Overall ↑EditReward PF ↑EditReward CS ↑EditReward Overall ↑
SFT63.9961.2463.5075.0047.5016.5046.3073.6572.0078.20
Offline distillation66.7364.6166.3578.0054.0020.5050.8080.2076.5078.45
CheckDistill73.3667.4272.3082.0060.5027.5056.6785.5084.1082.90

Table: Reward benchmark accuracy (%) for SFT, offline distillation, and CheckDistill.

W2&Q2: Downstream RL Comparison

Under the same DiffusionNFT setup, we conduct additional RL experiments with Qwen-Image-Edit using EditScore, SpatialReward, and FIRM-Edit as reward models.

CheckDistill provides a more effective reward signal for downstream RL. As shown in the table below, CheckDistill outperforms EditScore, SpatialReward, and FIRM-Edit on all three benchmarks.

ModelImgEdit-Bench↑REDEdit-Bench↑GEdit-EN↑
Qwen-Image-Edit4.294.108.18
EditScore + DiffusionNFT4.354.168.21
SpatialReward + DiffusionNFT4.324.238.26
FIRM-Qwen-Edit*4.42-7.84
FIRM-Edit + DiffusionNFT4.264.138.19
CheckDistill + DiffusionNFT4.464.268.28

Table: Downstream RL results with different reward models. FIRM-Qwen-Edit uses FIRM-Edit to guide RL on Qwen-Image-Edit. * indicates its scores are taken from [1].

We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

References:

  • [1] Zhao et al., Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation, arXiv 2026.
Reviewer_tDQqOpenReview ↗

Downstream RL comparison is addressed

Thanks for adding more downstream RL comparisons and I have raised the soundness score.

Thank you for raising the soundness score! We sincerely appreciate your valuable comments and the time and effort you put into reviewing our paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all three reviewers for their positive and detailed evaluations of our submission. We are pleased that the reviewers found CheckDistill "accurate and efficient" (tDQq, tg3v, wcJ8), appreciated the "clear problem formulation" (wcJ8), and recognized CheckDistill-40K with its instance-specific checklists as a "meaningful contribution" (tDQq, tg3v).

We would like to use this general response to summarize the changes in the revised manuscript, which we highlight in blue for easy reference.

Changes

  • We extended the RL training analysis with curves for all four reward dimensions and added a rollout analysis using unedited source images as baselines to assess whether the policy exploits preservation scores through no-ops or under-editing (Appendix I).
  • A new out-of-distribution (OOD) benchmark was introduced to provide further evidence of CheckDistill’s effectiveness under distribution shift (Appendix J).
  • We added an evaluation of the agreement between the automated annotations in CheckDistill-40K and human judgments (Appendix K).
  • We expanded CheckDistill’s inference efficiency evaluation to cover additional practical deployment metrics, including per-request generation time, throughput, and total runtime (Appendix L).

We believe these revisions substantially strengthen the empirical support for our findings. We thank all three reviewers for their constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.