acceptodds
Under review as a conference paper at ICLR 2027

MoRE-OPD: Mixture of Reward Experts with On-Policy Distillation for Image Editing

Abstract

Reliable evaluation is critical for image editing, supporting data curation, benchmarking, and reinforcement learning. However, diverse editing tasks require distinct criteria, leading to potential criterion entanglement in a shared evaluator. To address this challenge, we propose MoRE-OPD, a reward modeling framework that combines a Mixture of Reward Experts with privileged On-Policy Distillation. MoRE-OPD first learns category-specialized reward experts under task-specific evaluation protocols and then distills their heterogeneous evaluation capabilities into a compact universal evaluator. During OPD, specialized experts additionally access oracle-generated rationales as privileged context, enabling more effective supervision of student-generated predictions. Furthermore, we construct PointReward-Bench for reliable pointwise evaluation of image editing reward models, and apply MoRE-OPD to curate approximately 3.8 million high-quality samples for downstream optimization. Extensive experiments demonstrate that MoRE-OPD consistently outperforms state-of-the-art open-source image editing reward models across pointwise evaluation, preference alignment, and downstream reinforcement learning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Editing, Reward Modeling, On-Policy Distillation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_ztsVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper argues that image editing reward models suffer from criterion entanglement because different editing categories require different evaluation and preservation criteria. The authors train specialized reward experts for eight editing categories and consolidate their capabilities into a compact universal evaluator through privileged on-policy distillation. The method is evaluated on pointwise evaluation, preference ranking, data filtering, and downstream RL.

Critical strengths

The paper presents a complete and practically motivated reward modeling framework. It covers category-specific expert training, privileged on-policy distillation, preference adaptation, and downstream RL in a unified pipeline. The PointReward-Bench and downstream RL experiments suggest that the method improves both static reward evaluation and actual editing performance.

Critical weaknesses
  1. The necessity of training separate experts for different editing categories is not sufficiently established. Unlike domains such as mathematics and coding, the eight image editing categories share many fundamental capabilities, including instruction following, visual quality assessment, and preservation of unedited regions. It is unclear whether different editing categories truly require independent experts. Category conditioning, category-specific prompts, adapters, or a sufficiently expressive single model might already capture these distinctions. Therefore, it remains unclear whether the gains come from expert specialization itself, or from additional parameters, different data distributions, and more detailed annotation protocols.

  2. The reliability of category routing is not sufficiently analyzed. Multi-step edits and edits involving multiple objects may belong to several categories, but the paper does not study ambiguous categories, routing errors, or mixed-category examples.

Questions for authors

See Critical Weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer ztsV (Part 1/2)

We thank the reviewer for recognizing the practical significance of our work and the effectiveness of MoRE-OPD. We have significantly improved and revised the manuscript and conducted additional experiments to address concerns regarding expert specialization, category-aware routing, and ambiguous or mixed-category editing tasks, as detailed below.

Part 1: Expert Specialization and Category-Specific Adapters (W1)

Response: We thank the reviewer for the comment. We have conducted controlled comparisons between a single category-conditioned teacher and multiple specialized teachers, along with ablations on the number of experts, to assess the necessity of expert specialization. We also clarify the role of category-specific LoRA adapters in our framework. We have added these experiments to Table 2 and Section 4.5 of the revised manuscript, as summarized in Table D-1.

Table D-1. Ablation study on PointReward-Bench, corresponding to Table 2 in Section 4.5 (Ablation Studies) of the revised manuscript. Teachers (Qwen3.6-27B) use category-specific protocols RcR_c and are evaluated without privileged rationales. All OPD students (Qwen3.5-9B) start from the same SFT checkpoint and use the universal protocol RuniR_{\mathrm{uni}} during training and evaluation.

ModelMethodAcc. (%)TPFPTNFN
Qwen3.7-PlusOracle83.753236834762
Teacher (27B)1 Teacher (8 Categories Each)77.3825349366132
2 Teachers (4 Categories Each)79.5027352363112
4 Teachers (2 Categories Each)79.8827652363109
8 Teachers (1 Category Each)82.123318932654
Student (9B)Initial Student (SFT)80.383199132466
OPD w/o Privilege (8 Teachers)75.7535116025534
Privileged OPD (1 Teacher)80.123037733882
Privileged OPD (2 Teachers)80.502906135495
Privileged OPD (4 Teachers)81.503016435184
Privileged OPD (8 Teachers, Random Routing)80.883037134482
MoRE-OPD (8 Teachers, Ours)82.622954936690

Effectiveness of Multi-Teacher Specialization. To distinguish the contribution of expert specialization from that of category conditioning and category-specific prompts, we compare configurations with 1, 2, 4, and 8 teachers. All configurations use the same Qwen3.6-27B backbone, overall data pool, and category-specific protocols RcR_c. The single teacher handles all eight categories with their corresponding evaluation criteria, while multiple teachers specialize in assigned categories. For fewer teachers, we group semantically similar categories: four teachers each handle two categories, and two teachers each handle four categories. All configurations supervise the same initial 9B student through identical privileged OPD with the same oracle rationales.

As the number of teachers increases from 1 to 2, 4, and 8, teacher accuracy improves from 77.38% to 79.50%, 79.88%, and 82.12%, while student accuracy increases from 80.12% to 80.50%, 81.50%, and 82.62%. In particular, eight specialized teachers improve student accuracy by 2.50 percentage points over the single-teacher control. These results indicate that a single 27B teacher, even with explicit category conditioning, detailed prompts, and the same privileged rationales, is insufficient to capture category-dependent evaluation distinctions as effectively as multiple specialized teachers. It supports the necessity of multiple experts for providing the stronger supervision and improved distillation performance.

Category-Specific Adapters. We construct our specialized teachers using separate LoRA adapters on the same pretrained Qwen3.6-27B backbone. Each adapter is independently trained on its assigned editing categories with the corresponding evaluation protocols RcR_c. Thus, category-specific adapters are already part of our expert construction rather than an alternative to it.

However, using these adapters directly at inference would require category-specific adapter selection. Our privileged OPD instead consolidates their specialized evaluation capabilities into a single 9B student under the universal protocol RuniR_{\mathrm{uni}}, eliminating the need for multiple experts or adapter switching at inference.

AuthorsOpenReview ↗

Response to Reviewer ztsV (Part 2/2)

Part 2: Category Routing and Mixed-Category Edits (W2)

Response: We thank the reviewer for the comment. We clarify how MoRE-OPD handles category routing during training and inference, as well as ambiguous and mixed-category editing instructions. To address the concern that category ambiguity may lead to incorrect evaluation judgments, we analyze the impact of category misclassification on success/failure decisions. We have also conducted a category-wise analysis to assess the effectiveness of MoRE-OPD across different editing tasks, particularly multi-step edits. We have added the category-wise results to Appendix E of the revised manuscript.

Reliability of Category Routing. During OPD, we route samples to specialized teachers using dataset-provided category labels, without requiring a separate category classifier. At inference, the universal 9B student implicitly identifies editing types under the universal evaluation protocol RuniR_{\mathrm{uni}}, without explicit expert routing or adapter selection.

To examine the relationship between category misclassification and evaluation accuracy, we modify the inference output format to additionally require the student to predict one of the eight editing categories alongside its original evaluation scores and success/failure judgment, without retraining. The student correctly identifies 670 of 800 categories (83.75%). Among the 130 misclassified samples, 110 success/failure judgments remain correct (84.62%). These results indicate that incorrect category predictions do not necessarily lead to incorrect evaluation judgments. This suggests that the universal student has acquired evaluation capabilities across different editing categories, enabling it to assess IF, NC, and VQ even when the explicit category prediction is incorrect.

Handling Mixed-Category Edits. We assign instructions involving multiple editing operations to the Multi-Step Edits category. Its specialized expert evaluates each requested sub-task and the consistency across edits. Importantly, multiple objects do not necessarily imply multiple editing categories: removing two objects remains a removal task, whereas removing an object and modifying text constitutes a multi-step edit.

Our category-wise analysis on PointReward-Bench (Table D-2) shows that MoRE-OPD improves accuracy over the Initial Student in five of the eight editing categories, maintains the same accuracy in two, and decreases in one. Notably, on the 100-sample Multi-Step Edits subset, accuracy improves from 82% to 89%, representing the largest gain across all categories. These results support the effectiveness of our framework across diverse editing tasks, particularly compound editing instructions.

Table D-2. Category-wise evaluation on PointReward-Bench. Each category contains 100 samples. Δ\Delta denotes the accuracy difference in percentage points.

Editing CategoryInitialStudentMoRE-OPD (Ours)Δ\Delta (pp)
Acc. (%)TPFPTNFNAcc. (%)TPFPTNFN
Add Object or Element85431242390448462+5
Alter Background, Texture, or Style763364318722744524-4
Change Camera View or Perspective7625351217625351210
Edit People, Faces, or Character Attributes825015323824493890
Edit Text or Document Content85391346287357526+2
Modify Existing Object784013389843964510+6
Multi-Step Instruction (Compound Edits)82381444489365536+7
Remove Object or Element795115286814573612+2
Overall80.38319913246682.622954936690+2.25
Reviewer_ztsVOpenReview ↗

Review

Thank you for the thorough ablations. Scaling from 1 to 8 LoRA teachers under a fixed backbone and data pool (Table D-1) and the routing/multi-step analysis (Table D-2) directly address my concerns on expert specialization and category ambiguity. Please include these results in the revision. I am raising my Soundness score and updating my rating to 3 (Weak acceptance).

We sincerely thank you for your positive feedback and constructive suggestions. We are pleased that our additional experiments have addressed your concerns. We have incorporated the expert specialization ablations (Table D-1) into Section 4.5 (Table 2), and the category-wise evaluation results, including multi-step edits (Table D-2), into Appendix E of the revised manuscript. All additions and revisions have been highlighted in red in the revised manuscript.

Reviewer_LaRHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoRE-OPD, a reward modeling framework for image editing that aims to alleviate the entanglement of evaluation criteria across different editing tasks. The method trains eight 27B reward experts using category-specific evaluation protocols and distills their capabilities into a unified 9B student model through On-Policy Distillation with privileged information. Additionally, the paper introduces PointReward-Bench, containing 800 samples, and uses the proposed model to filter approximately 3.8 million image editing samples. Experiments demonstrate the effectiveness of the framework in reward evaluation and downstream reinforcement learning.

Critical strengths
  1. The paper is well organized, with a clear presentation of the methodology, making it easy to follow.
  2. The research problem is practically meaningful. Different image editing tasks have varying requirements for instruction following, content preservation, and visual quality. The paper addresses these differences through category-specific evaluation protocols and reward experts, and further integrates their capabilities into a unified student model, forming a clear technical framework.
  3. The experimental evaluation is relatively comprehensive. The paper evaluates the model on single-edit success prediction, preference alignment, and downstream reinforcement learning tasks. Positive results across multiple external benchmarks and downstream tasks provide some evidence of the practical effectiveness of the proposed method.
Critical weaknesses
  1. The paper primarily relies on automatic evaluation metrics and model-based judgments to assess downstream editing performance, without conducting human evaluation studies (user studies).
  2. Qwen3.7-Plus is used both to generate training supervision signals and privileged explanations and as the final judge in downstream benchmarks. This setup may introduce evaluation bias because the training supervision source and the evaluation model share preferences, potentially overestimating the advantages of the proposed method. Evaluation with additional independent judge models would help verify the reliability of the experimental conclusions.
  3. The downstream experiments use a model that has undergone additional preference training, but the paper does not report the performance of the initial SFT student model after undergoing the same preference training procedure. Therefore, it remains unclear to what extent the downstream improvements can be attributed to the proposed MoRE-OPD framework rather than the additional preference training. Including this baseline would allow for a more accurate assessment of the method’s contribution.
  4. The integration of multiple experts and On-Policy Distillation has already been explored in prior work such as MOPD. Similarly, the use of additional context to guide student models has been investigated in OPCD and related studies on privileged information distillation. Furthermore, Z-Reward has explored teacher–student distillation for visual reward models and includes comparisons with OPD. Therefore, the paper should more clearly distinguish its contributions from these existing approaches and demonstrate the unique benefits of combining these mechanisms for image editing reward modeling.
  5. Although PointReward-Bench is presented as one of the main contributions, the paper provides limited details about its construction process, data sources, task category distribution, and annotation quality control. In addition, representative examples from the benchmark are not visualized. More detailed descriptions, statistical analyses, and qualitative examples would improve the transparency and reproducibility of the benchmark.
  6. Under controlled conditions with the same teacher model size, training data, category-specific evaluation protocols, and oracle explanations, how much improvement do the eight category-specific experts provide compared with a single general-purpose teacher model? This comparison is important for validating the necessity of the multi-expert architecture.
  7. Figure 3: I suggest directly labeling the method names in each subfigure to make it easier for readers to identify and compare different methods without repeatedly referring to the figure caption.
  8. Figure 5: The editing instruction is simply “Make the image brighter,” without specifying the desired degree of brightness enhancement or target exposure level. Therefore, different levels of brightness enhancement may all satisfy the instruction, and the results produced by other methods may also be reasonable. A stronger brightness increase alone is insufficient to demonstrate better instruction following by MoRE-OPD. If the authors consider the other results to be overexposed, they should identify specific visual artifacts, such as highlight clipping or loss of details, to support this qualitative comparison.
Questions for authors
  1. What is the rationale behind choosing eight category-specific experts? Is this number determined by the natural categorization of image editing tasks, differences in evaluation criteria, or empirical design choices? Additionally, how does varying the number of experts affect model performance? Further clarification of the design rationale and relevant ablation studies would be helpful.
  2. Table 2: Why does OPD without privileged information achieve lower accuracy than the initial student model? The authors are encouraged to provide an explanation for this observation.
11 comments
AuthorsOpenReview ↗

Response to Reviewer LaRH (Part 1/7)

We thank the reviewer for recognizing the clarity of our methodology and the comprehensive evaluation of MoRE-OPD across multiple tasks. We have significantly improved and revised the manuscript and conducted additional experiments to address concerns regarding expert specialization, privileged OPD, preference adaptation, evaluation independence, and statistical robustness. We have also clarified the distinctions from prior work, PointReward-Bench construction, qualitative comparisons, and experimental presentation, as detailed below.

Part 1: Human-Annotated Evaluation and Independent Multi-Seed RL Evaluation (W1, W2)

Response to W1: Human-Annotated Evaluation. Our evaluation framework incorporates human judgments to assess the alignment of reward models with human preferences. Specifically, PointReward-Bench is constructed using annotations from three independent human evaluators, with unanimous agreement on 730 out of 800 samples (91.25%). Furthermore, our preference evaluation measures agreement with reference preference labels on established benchmarks, including human-annotated preferences. These evaluations provide direct evidence of the reward model's alignment with human judgments.

For downstream RL, we evaluate editing performance on two publicly available and widely used image editing benchmarks, GEdit-Bench and ImgEdit-Bench, following their established evaluation protocols. To strengthen evaluation independence and robustness, we have additionally conducted three independent RL training runs, each evaluated with three generation seeds using an independent non-Qwen judge (Gemini 3.8 Flash).

As reported in Appendix C (Table C-1), MoRE-OPD achieves the highest mean scores across all GEdit-Bench and ImgEdit-Bench metrics, consistent with the original Qwen-based evaluation. By following established evaluation protocols on widely used public benchmarks and introducing an independent judge and multiple training runs, we provide reliable and reproducible evidence of MoRE-OPD's downstream effectiveness.

Table C-1. Independent-judge evaluation on GEdit-Bench and ImgEdit-Bench. Results are mean ±\pm standard deviation over three RL training runs, each with three generation seeds. The non-RL baseline uses three generation seeds. The last row reports paired differences between Ours and the strongest baseline across training runs.

Reward ModelGEdit-BenchImgEdit-Bench
QSC↑Q_{\mathrm{SC}} \uparrowQPQ↑Q_{\mathrm{PQ}} \uparrowQO↑Q_{\mathrm{O}} \uparrowOverall ↑\uparrow
None (Klein)6.30±0.0246.30 \pm 0.0245.94±0.0235.94 \pm 0.0236.07±0.0196.07 \pm 0.0193.43±0.0073.43 \pm 0.007
EditScore6.33±0.0156.33 \pm 0.0155.98±0.0235.98 \pm 0.0236.10±0.0126.10 \pm 0.0123.44±0.016‾\underline{3.44 \pm 0.016}
EditReward6.31±0.0066.31 \pm 0.0065.93±0.0165.93 \pm 0.0166.07±0.0116.07 \pm 0.0113.42±0.0053.42 \pm 0.005
SpatialReward6.35±0.005‾\underline{6.35 \pm 0.005}5.99±0.031‾\underline{5.99 \pm 0.031}6.12±0.014‾\underline{6.12 \pm 0.014}3.44±0.0083.44 \pm 0.008
RubricRM6.30±0.0136.30 \pm 0.0135.93±0.0185.93 \pm 0.0186.07±0.0126.07 \pm 0.0123.43±0.0053.43 \pm 0.005
RationalRewards6.31±0.0076.31 \pm 0.0075.94±0.0175.94 \pm 0.0176.07±0.0066.07 \pm 0.0063.44±0.0093.44 \pm 0.009
Ours6.37±0.009\mathbf{6.37 \pm 0.009}6.05±0.013\mathbf{6.05 \pm 0.013}6.16±0.012\mathbf{6.16 \pm 0.012}3.45±0.012\mathbf{3.45 \pm 0.012}
Δ\Delta (Ours −- Best)+0.0200±0.0086+0.0200 \pm 0.0086+0.0536±0.0314+0.0536 \pm 0.0314+0.0371±0.0187+0.0371 \pm 0.0187+0.0043±0.0124+0.0043 \pm 0.0124

Response to W2: Independent Multi-Seed RL Evaluation. We thank the reviewer for the suggestion. We have re-evaluated all methods using an independent non-Qwen judge (Gemini 3.8 Flash) and conducted three independent RL training runs. Consistent with the original Qwen-based evaluation, MoRE-OPD achieves the highest mean scores across all downstream RL metrics. These results have been added to Appendix C of the revised manuscript, as reported in Table C-1.

Experimental setup and robustness. All methods share the same base editing model, training data, and RL settings, differing only in the reward model. We use three training seeds (0, 42, 2026), each evaluated with three generation seeds (42, 1819, 2027). We follow the original GEdit-Bench and ImgEdit-Bench protocols, replacing Qwen3.7-Plus with Gemini 3.8 Flash.

We report the mean and standard deviation across training runs after averaging over generation seeds, along with paired differences against the strongest competing method. The baseline without RL is evaluated using the same three generation seeds. The results confirm that the observed improvements persist under an independent judge and across multiple training runs.

AuthorsOpenReview ↗

Response to Reviewer LaRH (Part 2/7)

Part 2: Controlled Preference Adaptation and Downstream RL (W3)

Response: We thank the reviewer for the suggestion. We have added a preference-adapted Initial Student baseline to isolate the contribution of MoRE-OPD from additional preference training. The results show that MoRE-OPD outperforms this baseline in both preference evaluation and downstream RL. We have added these experiments to Appendix D of the revised manuscript, as reported in Tables C-2 and C-3.

Table C-2. Preference evaluation of the Initial Student (SFT) and MoRE-OPD under identical preference adaptation settings.

MethodEditScore-BenchEditReward-BenchMMRB2
PFCons.Ovrl.Avg.K2K3K4Avg.Acc.
Initial Student (SFT)85.4984.4982.7184.0877.4440.0010.1441.3071.70
MoRE-OPD (Ours)87.8287.4284.7386.4679.7046.6716.2246.4073.80

Table C-3. Downstream RL comparison using the preference-adapted Initial Student and MoRE-OPD as reward models. Results are reported as mean ±\pm standard deviation across three independent RL training runs, following the evaluation procedure in Appendix C.

Reward ModelGEdit-BenchImgEdit-Bench
QSCQ_{\mathrm{SC}}QPQQ_{\mathrm{PQ}}QOQ_{\mathrm{O}}Overall
Initial Student (SFT)6.35±0.0116.35 \pm 0.0115.98±0.0215.98 \pm 0.0216.11±0.0126.11 \pm 0.0123.44±0.0033.44 \pm 0.003
MoRE-OPD (Ours)6.37±0.0096.37 \pm 0.0096.05±0.0136.05 \pm 0.0136.16±0.0126.16 \pm 0.0123.45±0.0123.45 \pm 0.012

Matched preference adaptation. We apply identical preference data, LoRA configurations, scalar reward heads, and optimization settings to both models, differing only in their initialization checkpoints. MoRE-OPD outperforms the adapted Initial Student by 2.38, 5.10, and 2.10 percentage points on EditScore-Bench, EditReward-Bench, and MMRB2-Edit, respectively. Consistent with the improvements observed in pointwise evaluation, these results further demonstrate that MoRE-OPD enhances the model's evaluation capabilities, rather than merely benefiting from preference adaptation via LoRA.

Controlled downstream RL comparison. We further compare both adapted models under identical RL training and evaluation settings. Following Appendix C, we conduct three independent RL training runs, each evaluated with three generation seeds using the independent Gemini 3.8 Flash judge. We first average the scores over generation seeds within each run, then report the mean and standard deviation across the three training runs. As shown in Table C-3, MoRE-OPD achieves higher mean scores across all GEdit-Bench and ImgEdit-Bench metrics, confirming that its benefits extend to downstream RL.

AuthorsOpenReview ↗

Response to Reviewer LaRH (Part 3/7)

Part 3: Distinctions from Prior Distillation and Reward Modeling Methods (W4)

Response: We thank the reviewer for the suggestion. We have clarified our distinctions from MOPD, OPCD, and Z-Reward in Sections 2.1 and 2.2 of the revised manuscript.

Although MOPD and OPCD have explored multi-teacher and context-conditioned on-policy distillation, MoRE-OPD differs fundamentally in the problem addressed, the mechanisms employed, and the training framework. While these methods primarily focus on language model capability distillation, MoRE-OPD specifically addresses criterion entanglement in image editing reward modeling. We train category-specialized reward experts using distinct evaluation protocols and combine category-aligned teacher supervision with privileged oracle rationales to distill their capabilities into a unified student.

Furthermore, MoRE-OPD differs from Z-Reward in both the research problem and methodology. While Z-Reward focuses on text-to-image reward modeling through score distribution distillation, MoRE-OPD addresses criterion entanglement in image editing reward modeling through category-specialized supervision and privileged on-policy distillation.

Reviewer_3zHrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoRE-OPD, a framework for image editing reward modeling that aims to address conflicting evaluation criteria across editing categories. Eight category-specialized 27B experts are trained using oracle annotations and task-specific protocols. Their capabilities are then distilled into a universal 9B evaluator through on-policy distillation, with oracle-generated textual rationales provided exclusively to the teachers as privileged context. A subsequent preference adaptation stage adds scalar reward heads and uses LoRA with pairwise supervision.

The paper also introduces PointReward-Bench, an 800-example human-annotated benchmark covering eight editing categories, and reports filtering approximately 3.8 million editing samples. Evaluation covers pointwise success classification, three preference benchmarks, and downstream reinforcement learning. The reported results show strong pointwise performance and promising preference and downstream improvements.

Critical strengths
  1. The paper addresses a practically important distinction between judging whether an edit succeeds and ranking competing edits. PointReward-Bench provides a useful evaluation setting with balanced category coverage and majority-vote labels from three annotators. On this benchmark, MoRE-OPD achieves 82.62% accuracy compared with 75.12% for the strongest external baseline, RationalRewards, while substantially reducing false positives. This is relevant to applications such as training-data filtering.

  2. The framework consolidates multiple specialized teachers into a single 9B evaluator that requires neither the expert ensemble nor privileged rationales at inference. The evaluation extends beyond pointwise classification to preference prediction and downstream RL. In particular, the RL comparisons hold the editing model, training data, and optimization configuration fixed, providing a useful controlled comparison of reward models.

Critical weaknesses
  1. The experiments do not sufficiently establish the contribution of category-specialized teachers, which is central to the proposed explanation of criterion entanglement. Table 2 demonstrates the importance of privileged rationales within the proposed pipeline, but does not compare against distillation from a single teacher with access to the same category-specific protocols and privileged information. The initial student already achieves 80.38% accuracy, compared with 82.62% after the full procedure, while OPD without privileged information drops to 75.75%. These results support the usefulness of privileged supervision, but leave the necessity of multiple specialized experts unresolved. Moreover, no category-level analysis directly demonstrates reduced criterion entanglement, and the preference experiments lack an initial-student baseline receiving the same preference adaptation. This limits attribution of the reported gains to the central methodological contribution.

  2. The downstream validation is insufficiently independent of the supervision source. Qwen3.7-Plus supplies the oracle annotations and additional NC preference labels, and Section 6.1 also uses it to replace the default judges on both downstream benchmarks. Although all methods are evaluated by the same judge, the proposed reward model is specifically trained to reproduce supervision from that model, creating a risk of correlated evaluation preferences. The margins over the strongest competing RL results are modest: 0.10 on GEdit-Bench overall and 0.05 on ImgEdit-Bench, without reported uncertainty or variation across training runs. The human-annotated reward benchmarks provide valuable complementary evidence, but do not resolve whether these downstream gains reflect improvements recognized by independent evaluators.

Questions for authors

The following questions concern evidence needed to assess the current claims.

  1. What improvement is specifically attributable to using eight specialized teachers? Can you provide a comparison with a single 27B teacher trained on the pooled data and conditioned on the corresponding category-specific protocol and oracle rationale? This would help distinguish teacher specialization from the benefits of privileged supervision.

  2. How much of the preference improvement remains when the initial SFT student receives exactly the same LoRA adaptation, scalar heads, and preference supervision? This control would clarify whether the OPD stage is responsible for the downstream preference gains.

  3. Do the RL improvements persist under the original benchmark judges, an independent judge, or a blinded human comparison? Please also provide uncertainty estimates for the reported differences and clarify the number of independent RL training runs.

  4. Can you report category-wise results for the initial student and MoRE-OPD, together with annotator agreement on PointReward-Bench? Table 2 shows that false positives decrease from 91 to 49 while false negatives increase from 66 to 90. A category-level breakdown and sensitivity analysis of the success thresholds would help determine whether the improvement reflects better category-dependent judgments or primarily a more conservative acceptance policy.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 3zHr (Part 1/4)

We thank the reviewer for recognizing the value of PointReward-Bench, our multi-teacher distillation framework, and the controlled downstream RL experiments. We have significantly improved and revised the manuscript and conducted additional experiments to address the concerns regarding expert specialization, category-wise evaluation, preference adaptation, evaluation independence, multi-seed robustness, and annotation reliability, as detailed below. For clarity, we divide your first concern into three aspects, labeled W1.1–W1.3 in our responses: expert specialization (W1.1), category-wise analysis (W1.2), and controlled preference adaptation (W1.3), respectively. These labels are introduced solely to organize our responses.

Part 1: Expert Specialization under Matched Privileged Supervision (W1.1, Q1)

Response: We thank the reviewer for the suggestion. We have conducted additional single- and multi-teacher ablations under matched privileged supervision to verify the effectiveness of expert specialization. We have added these results to Table 2 and Section 4.5 of the revised manuscript (“Effectiveness of Multi-Teacher Specialization”), as shown in Table B-1.

Table B-1. Ablation study on PointReward-Bench, corresponding to Table 2 in Section 4.5 (Ablation Studies) of the revised manuscript. Teachers (Qwen3.6-27B) use category-specific protocols RcR_c and are evaluated without privileged rationales. All OPD students (Qwen3.5-9B) start from the same SFT checkpoint and use the universal protocol RuniR_{\mathrm{uni}} during training and evaluation.

ModelMethodAcc. (%)TPFPTNFN
Qwen3.7-PlusOracle83.753236834762
Teacher (27B)1 Teacher (8 Categories Each)77.3825349366132
2 Teachers (4 Categories Each)79.5027352363112
4 Teachers (2 Categories Each)79.8827652363109
8 Teachers (1 Category Each)82.123318932654
Student (9B)Initial Student (SFT)80.383199132466
OPD w/o Privilege (8 Teachers)75.7535116025534
Privileged OPD (1 Teacher)80.123037733882
Privileged OPD (2 Teachers)80.502906135495
Privileged OPD (4 Teachers)81.503016435184
Privileged OPD (8 Teachers, Random Routing)80.883037134482
MoRE-OPD (8 Teachers, Ours)82.622954936690

Experimental setup. We compare 1, 2, 4, and 8 Qwen3.6-27B teachers, each handling 8, 4, 2, and 1 editing categories, respectively. We group semantically related categories for the 2- and 4-teacher settings. All configurations use the same data pool and category-specific protocols RcR_c. In particular, the single teacher learns all eight categories using their corresponding protocols. Each configuration supervises the same initial 9B student through identical privileged OPD with the same oracle rationales. Teachers are evaluated under RcR_c without privileged rationales, while students use the universal protocol RuniR_{\mathrm{uni}}.

Results. As shown in Table B-1, teacher accuracy increases from 77.38% to 79.50%, 79.88%, and 82.12%, while student accuracy improves from 80.12% to 80.50%, 81.50%, and 82.62% as the number of teachers increases. Compared with the single-teacher setting, eight specialized teachers improve teacher and student accuracy by 4.74 and 2.50 percentage points, respectively. These results demonstrate that expert specialization provides additional benefits beyond privileged supervision alone, supporting its role in alleviating criterion entanglement.

AuthorsOpenReview ↗

Response to Reviewer 3zHr (Part 2/4)

Part 2: Controlled Preference Adaptation and Downstream RL (W1.3, Q2)

Response: We thank the reviewer for the suggestion. We have applied identical preference adaptation to the Initial Student and MoRE-OPD to isolate the contribution of the OPD stage. The results show that MoRE-OPD consistently outperforms the Initial Student in preference evaluation and downstream RL. We have added these experiments to Appendix D of the revised manuscript, as reported in Tables B-2 and B-3.

Table B-2. Preference evaluation of the Initial Student (SFT) and MoRE-OPD under identical preference adaptation settings.

MethodEditScore-BenchEditReward-BenchMMRB2
PFCons.Ovrl.Avg.K2K3K4Avg.Acc.
Initial Student (SFT)85.4984.4982.7184.0877.4440.0010.1441.3071.70
MoRE-OPD (Ours)87.8287.4284.7386.4679.7046.6716.2246.4073.80

Table B-3. Downstream RL comparison using the preference-adapted Initial Student and MoRE-OPD as reward models. Results are reported as mean ±\pm standard deviation across three independent RL training runs, following the evaluation procedure in Appendix C.

Reward ModelGEdit-BenchImgEdit-Bench
QSCQ_{\mathrm{SC}}QPQQ_{\mathrm{PQ}}QOQ_{\mathrm{O}}Overall
Initial Student (SFT)6.35±0.0116.35 \pm 0.0115.98±0.0215.98 \pm 0.0216.11±0.0126.11 \pm 0.0123.44±0.0033.44 \pm 0.003
MoRE-OPD (Ours)6.37±0.0096.37 \pm 0.0096.05±0.0136.05 \pm 0.0136.16±0.0126.16 \pm 0.0123.45±0.0123.45 \pm 0.012

Controlled preference adaptation. We apply identical preference training data, LoRA configurations, scalar reward heads, and optimization settings to the Initial Student and MoRE-OPD. Both adapted models are evaluated on EditScore-Bench, EditReward-Bench, and MMRB2-Edit under identical evaluation settings.

Preference evaluation results. MoRE-OPD achieves 86.46%, 46.40%, and 73.80% on EditScore-Bench, EditReward-Bench, and MMRB2-Edit, respectively, outperforming the preference-adapted Initial Student by 2.38, 5.10, and 2.10 percentage points. Consistent with the improvements observed in pointwise evaluation, MoRE-OPD achieves higher performance across all three preference benchmarks under identical adaptation settings. These consistent improvements further demonstrate that MoRE-OPD enhances the model's evaluation capabilities, rather than merely benefiting from preference adaptation via LoRA.

Downstream RL results. We further compare both preference-adapted models as reward models under identical RL settings. Following Appendix C, we conduct three independent RL training runs, each evaluated with three generation seeds using Gemini 3.8 Flash.

As shown in Table B-3, MoRE-OPD achieves higher mean scores than the Initial Student across all reported metrics on GEdit-Bench and ImgEdit-Bench. Consistent with the pointwise and preference evaluation results, these findings further demonstrate that the evaluation capabilities acquired through MoRE-OPD also translate into improved downstream RL performance.

AuthorsOpenReview ↗

Response to Reviewer 3zHr (Part 3/4)

Part 3: Independent Evaluation and Multi-Seed RL Robustness (W2, Q3)

Response: We thank the reviewer for the question. To address potential evaluation bias arising from the shared Qwen-based supervision and evaluation, we have conducted additional multi-seed RL experiments using an independent non-Qwen judge (Gemini 3.8 Flash). The independent-judge results are consistent with the original evaluation, with MoRE-OPD maintaining the best mean performance across multiple training seeds. The results have been added to Appendix C of the revised manuscript and are reported in Table B-4.

Table B-4. Independent-judge evaluation on GEdit-Bench and ImgEdit-Bench. Results are mean ±\pm standard deviation over three RL training runs, each with three generation seeds. The non-RL baseline uses three generation seeds. The last row reports paired differences between Ours and the strongest baseline across training runs.

Reward ModelGEdit-BenchImgEdit-Bench
QSC↑Q_{\mathrm{SC}} \uparrowQPQ↑Q_{\mathrm{PQ}} \uparrowQO↑Q_{\mathrm{O}} \uparrowOverall ↑\uparrow
None (Klein)6.30±0.0246.30 \pm 0.0245.94±0.0235.94 \pm 0.0236.07±0.0196.07 \pm 0.0193.43±0.0073.43 \pm 0.007
EditScore6.33±0.0156.33 \pm 0.0155.98±0.0235.98 \pm 0.0236.10±0.0126.10 \pm 0.0123.44±0.016‾\underline{3.44 \pm 0.016}
EditReward6.31±0.0066.31 \pm 0.0065.93±0.0165.93 \pm 0.0166.07±0.0116.07 \pm 0.0113.42±0.0053.42 \pm 0.005
SpatialReward6.35±0.005‾\underline{6.35 \pm 0.005}5.99±0.031‾\underline{5.99 \pm 0.031}6.12±0.014‾\underline{6.12 \pm 0.014}3.44±0.0083.44 \pm 0.008
RubricRM6.30±0.0136.30 \pm 0.0135.93±0.0185.93 \pm 0.0186.07±0.0126.07 \pm 0.0123.43±0.0053.43 \pm 0.005
RationalRewards6.31±0.0076.31 \pm 0.0075.94±0.0175.94 \pm 0.0176.07±0.0066.07 \pm 0.0063.44±0.0093.44 \pm 0.009
Ours6.37±0.009\mathbf{6.37 \pm 0.009}6.05±0.013\mathbf{6.05 \pm 0.013}6.16±0.012\mathbf{6.16 \pm 0.012}3.45±0.012\mathbf{3.45 \pm 0.012}
Δ\Delta (Ours −- Best)+0.0200±0.0086+0.0200 \pm 0.0086+0.0536±0.0314+0.0536 \pm 0.0314+0.0371±0.0187+0.0371 \pm 0.0187+0.0043±0.0124+0.0043 \pm 0.0124

Experimental setup. The original Table 3 reports a single RL training run (seed 42) evaluated with generation seed 42. We additionally perform two independent training runs, resulting in three training seeds (0, 42, 2026). Each checkpoint is evaluated using three generation seeds (42, 1819, 2027) under the original benchmark protocols, replacing Qwen3.7-Plus with Gemini 3.8 Flash. All training settings are identical across methods, with only the reward model changed.

Results and robustness. We average scores over generation seeds and report the mean and standard deviation across training runs. To quantify the uncertainty of the reported improvements, we also report paired differences against the strongest competing method. For the FLUX.2 [klein] baseline without RL, statistics are computed across three generation seeds.

Consistent with the original Qwen-based evaluation, MoRE-OPD achieves the highest mean scores across all GEdit-Bench and ImgEdit-Bench metrics under the independent judge. These results support the robustness of our downstream RL improvements across different judges and training runs.

Reviewer_h89GOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MoRE-OPD, a reward modeling framework for instruction-guided image editing. Motivated by conflicting evaluation criteria across edit types, the authors train eight category-specialized 27B experts with oracle labels, then distill them into a 9B universal student via privileged on-policy distillation (experts see oracle textual rationales; the student does not). A preference stage follows. They also introduce PointReward-Bench (800 human-labeled pointwise success/failure examples) and filter about 3.8M editing pairs for downstream use. Experiments cover pointwise classification, preference benchmarks, and Flow-GRPO fine-tuning of FLUX.2.

Critical strengths
  1. Pointwise evaluation fills a real gap. Most editing reward benches focus on pairwise preference. PointReward-Bench asks for an absolute success/failure label per edit. That is useful for data curation and for absolute reward signals, and it supports the later large-scale filtering story. This is a main reason I lean accept.
  2. The pipeline is complete and the pointwise numbers are strong. The path from specialized experts and privileged distillation to preference heads, data filtering, and RL is clear. On Table 1, PointReward accuracy reaches 82.62, above recent baselines, with few false positives. That matches the claim that the model is good at rejecting failed edits. The writing is easy to follow.
Critical weaknesses
  1. The oracle / evaluation loop is not fully separated. The annotation oracle, some preference labels, and the downstream RL judge all use the same Qwen family. On the preference / RL tables, gains over the best competing rewards are small. The paper has not yet shown whether the advantage remains under an independent human or non-Qwen automatic judge.
  2. The mixture / disentanglement contribution is not fully isolated. In Table 2, the full method is only a small gain over the initial student, while removing privileged rationales drops accuracy a lot and raises false positives. Privileged oracle text therefore appears to drive much of the gain. The paper does not report a single-protocol single-model control, or random routing versus category routing. Clearer controls would better support the mixture-based disentanglement stressed in the title. These points do not outweigh the benchmark contribution and the strong pointwise results for me, but they keep the paper at weak accept rather than a stronger accept. An independent judge and mixture/routing/protocol ablations, plus annotator agreement on PointReward-Bench, would make me more confident.
Questions for authors
  1. If downstream RL and preference evaluation use independent human preferences, or at least a non-Qwen automatic judge, does MoRE-OPD still beat the strongest baselines?
  2. Please ablate a 9B model trained only with the universal protocol, and OPD with shuffled category routing versus true category routing. How large are the gaps, and would that change how the mixture / disentanglement claim is framed?
  3. What is the inter-annotator agreement on PointReward-Bench? For continuous baselines, can you re-report accuracy with a fixed or validation-chosen threshold instead of exhaustive search on the test set?
  4. At inference the universal protocol must infer the edit type. What is that inference accuracy, and how do decisions change when the type is wrong?
  5. Does Table 3 include multiple seeds or variance? Are the small gaps stable?
4 comments
AuthorsOpenReview ↗

Response to Reviewer h89G (Part 1/4)

We thank the reviewer for recognizing the value of PointReward-Bench and the effectiveness of MoRE-OPD. We have improved and revised the manuscript and conducted additional experiments to address the concerns regarding evaluation independence, expert specialization, category-aware routing, robustness to edit-type misclassification, annotation reliability, and downstream robustness, as detailed below.

Part 1: Independent Evaluation and Multi-Seed RL Robustness (W1, Q1, Q5)

Response: We thank the reviewer for the suggestions. We address evaluation independence and robustness through human-annotated preference benchmarks and additional multi-seed RL experiments with an independent non-Qwen judge. The results are consistent with those obtained using the original Qwen-based judge, with MoRE-OPD maintaining the best overall performance under the independent judge across multiple training runs. We have added the new RL results to Appendix C of the revised manuscript, as reported in Table A-1.

Table A-1. Independent-judge evaluation on GEdit-Bench and ImgEdit-Bench. Results are mean ±\pm standard deviation over three RL training runs, each with three generation seeds. The non-RL baseline uses three generation seeds. The last row reports paired differences between Ours and the strongest baseline across training runs.

Reward ModelGEdit-BenchImgEdit-Bench
QSC↑Q_{\mathrm{SC}} \uparrowQPQ↑Q_{\mathrm{PQ}} \uparrowQO↑Q_{\mathrm{O}} \uparrowOverall ↑\uparrow
None (Klein)6.30±0.0246.30 \pm 0.0245.94±0.0235.94 \pm 0.0236.07±0.0196.07 \pm 0.0193.43±0.0073.43 \pm 0.007
EditScore6.33±0.0156.33 \pm 0.0155.98±0.0235.98 \pm 0.0236.10±0.0126.10 \pm 0.0123.44±0.016‾\underline{3.44 \pm 0.016}
EditReward6.31±0.0066.31 \pm 0.0065.93±0.0165.93 \pm 0.0166.07±0.0116.07 \pm 0.0113.42±0.0053.42 \pm 0.005
SpatialReward6.35±0.005‾\underline{6.35 \pm 0.005}5.99±0.031‾\underline{5.99 \pm 0.031}6.12±0.014‾\underline{6.12 \pm 0.014}3.44±0.0083.44 \pm 0.008
RubricRM6.30±0.0136.30 \pm 0.0135.93±0.0185.93 \pm 0.0186.07±0.0126.07 \pm 0.0123.43±0.0053.43 \pm 0.005
RationalRewards6.31±0.0076.31 \pm 0.0075.94±0.0175.94 \pm 0.0176.07±0.0066.07 \pm 0.0063.44±0.0093.44 \pm 0.009
Ours6.37±0.009\mathbf{6.37 \pm 0.009}6.05±0.013\mathbf{6.05 \pm 0.013}6.16±0.012\mathbf{6.16 \pm 0.012}3.45±0.012\mathbf{3.45 \pm 0.012}
Δ\Delta (Ours −- Best)+0.0200±0.0086+0.0200 \pm 0.0086+0.0536±0.0314+0.0536 \pm 0.0314+0.0371±0.0187+0.0371 \pm 0.0187+0.0043±0.0124+0.0043 \pm 0.0124

Preference Evaluation. Our preference evaluation uses three human-annotated benchmarks: EditScore-Bench, EditReward-Bench, and MMRB2-Edit, with human preferences as ground-truth labels. Qwen3.7-Plus is used only to supplement NC preference labels during training, not as the evaluation judge for these benchmarks. As reported in Table 1 of the manuscript, MoRE-OPD outperforms the strongest competing baseline in the aggregate metric of each benchmark: 86.46% vs. 79.87% on EditScore-Bench (average), 46.40% vs. 46.17% on EditReward-Bench (average), and 73.80% vs. 72.85% on MMRB2-Edit (accuracy).

Independent-Judge Evaluation and Multi-Seed Robustness. Table 3 in the original manuscript reports a single RL training run (seed 42) evaluated with generation seed 42. We additionally conduct two independent training runs, yielding three training seeds (0, 42, 2026). Each checkpoint is evaluated using three generation seeds (42, 1819, 2027) with an independent judge (Gemini 3.8 Flash), following the same benchmark protocols. The base editing model, training data, and RL settings are identical across methods, with only the reward model changed.

For each method, we average results across generation seeds and report the mean and standard deviation across training runs. We also report paired differences against the strongest competing method to quantify the uncertainty. The FLUX.2 [klein] baseline, which requires no RL training, is evaluated using the same three generation seeds.

Results. As shown in Table A-1, MoRE-OPD achieves the highest mean scores across all three GEdit-Bench metrics and the overall ImgEdit-Bench score under the independent Gemini judge. The paired performance differences are positive on average across all four metrics, while their standard deviations characterize the variability of these improvements across independent training runs. These results provide further evidence that the observed downstream RL advantages extend beyond the original Qwen-based evaluation and a single training run.

AuthorsOpenReview ↗

Response to Reviewer h89G (Part 2/4)

Part 2: Contributions of Expert Specialization, Routing, and Privileged Supervision (W2, Q2)

Response: We thank the reviewer for the suggestions. We have conducted additional ablation experiments comparing single- and multi-teacher distillation, random and category-aware routing, and OPD with and without privileged rationales to clarify their respective contributions to MoRE-OPD. Our results demonstrate the complementary benefits of expert specialization, category-aware routing, and privileged supervision. We have added these experiments to Table 2 and Section 4.5 of the revised manuscript, as summarized in Table A-2. The independent-judge evaluation and PointReward-Bench annotation agreement are addressed in our responses to W1 and Q3, respectively.

Table A-2. Ablation study on PointReward-Bench, corresponding to Table 2 in Section 4.5 (Ablation Studies) of the revised manuscript. Teachers (Qwen3.6-27B) use category-specific protocols RcR_c and are evaluated without privileged rationales. All OPD students (Qwen3.5-9B) start from the same SFT checkpoint and use the universal protocol RuniR_{\mathrm{uni}} during training and evaluation.

ModelMethodAcc. (%)TPFPTNFN
Qwen3.7-PlusOracle83.753236834762
Teacher (27B)1 Teacher (8 Categories Each)77.3825349366132
2 Teachers (4 Categories Each)79.5027352363112
4 Teachers (2 Categories Each)79.8827652363109
8 Teachers (1 Category Each)82.123318932654
Student (9B)Initial Student (SFT)80.383199132466
OPD w/o Privilege (8 Teachers)75.7535116025534
Privileged OPD (1 Teacher)80.123037733882
Privileged OPD (2 Teachers)80.502906135495
Privileged OPD (4 Teachers)81.503016435184
Privileged OPD (8 Teachers, Random Routing)80.883037134482
MoRE-OPD (8 Teachers, Ours)82.622954936690

Single-model, universal-protocol baseline. The Initial Student is a Qwen3.5-9B model trained through SFT on pooled oracle annotations under RuniR_{\mathrm{uni}}. It achieves 80.38% accuracy, compared with 82.62% for MoRE-OPD. Both use the same 9B architecture and universal protocol, demonstrating the benefit of MoRE-OPD without increasing student model capacity.

Effectiveness of Multi-Teacher Specialization. To distinguish the contribution of expert specialization from the benefit of privileged rationales, we compare configurations with 1, 2, 4, and 8 teachers, each responsible for 8, 4, 2, and 1 editing categories, respectively. All configurations use the same data pool and category-specific protocols RcR_c, and distill into the same initial 9B student using identical privileged OPD and oracle rationales. The single teacher handles all eight categories with their corresponding protocols.

We progressively merge semantically related categories. For four teachers, we group: (1) Add and Remove Object; (2) Modify Existing Object and Edit People; (3) Alter Background/Style and Change Camera View; and (4) Multi-Step Edits and Edit Text. For two teachers, we merge groups (1)–(2) and (3)–(4), respectively. Each sample is routed to the teacher responsible for its category during distillation.

As the number of teachers increases from 1 to 2, 4, and 8, teacher accuracy improves from 77.38% to 79.50%, 79.88%, and 82.12%, while student accuracy improves from 80.12% to 80.50%, 81.50%, and 82.62%. The eight-teacher configuration improves student accuracy by 2.50 percentage points under identical privileged supervision, demonstrating the additional benefit of expert specialization; the gain cannot be attributed to access to oracle rationales alone.

Category-matched versus random routing. We randomly assign each sample to one of the same eight experts while keeping privileged supervision unchanged. Random routing reduces accuracy from 82.62% to 80.88%, demonstrating the importance of matching expert supervision to editing categories. The result supports our multi-expert distillation strategy: the full gain depends on how the specialized experts supervise the student, and cannot be explained simply by using more teacher parameters.

Contribution of privileged rationales. Removing privileged rationales from the eight-teacher configuration reduces accuracy from 82.62% to 75.75% and increases false positives from 49 to 160. This confirms the importance of privileged supervision.

These controls strengthen the empirical basis for our mixture-based approach by showing that category-specialized, category-matched supervision provides additional gains when privileged information is held fixed. The resulting evaluation capabilities are consolidated into a single 9B student that requires neither multiple teachers nor privileged rationales at inference.

AuthorsOpenReview ↗

Response to Reviewer h89G (Part 3/4)

Part 3: Annotation Reliability and Validation-Based Threshold Selection (Q3)

Response: We thank the reviewer for the suggestions. We have added annotation agreement statistics to Appendix F and validation-based threshold selection experiments to Appendix G of the revised manuscript, with a reference in Section 4.1. The results confirm the reliability of human annotations and the advantage of MoRE-OPD under independently selected baseline thresholds.

Inter-annotator agreement. Three annotators independently label each of the 800 PointReward-Bench samples, with final labels determined by majority voting. They unanimously agree on 730 samples (91.25%), with a mean pairwise agreement of 94.17% and Fleiss' κ=0.8833\kappa=0.8833, indicating strong annotation consistency. The final labels comprise 385 successful and 415 failed edits.

Validation-based threshold selection. We construct an independent validation set of 400 samples (50 per category), annotated by three annotators using majority voting. For each continuous-score baseline, we select the threshold maximizing validation accuracy with a step size of 0.01 and apply it unchanged to the 800-sample test set. MoRE-OPD uses fixed criteria (IF ≥9\geq 9, NC ≥8\geq 8, VQ ≥6\geq 6), without threshold tuning.

As shown in Table A-3, RationalRewards achieves the highest accuracy among the baselines at 74.25%, which is 8.37 percentage points lower than MoRE-OPD (82.62%). Our advantage therefore persists when competing baselines select their thresholds independently of the test labels.

Table A-3. PointReward-Bench results for continuous-score baselines under test-optimal and validation-selected thresholds.

MethodThresholdVal. Acc. (%)Test-Optimal Acc. (%)Val-Selected Test Acc. (%)
EditScore7.4571.368.2567.38
SpatialReward8.4078.371.5071.38
EditReward0.9679.071.6270.38
RationalRewards0.7476.875.1274.25
RubricRM3.5179.573.3872.50
AuthorsOpenReview ↗

Response to Reviewer h89G (Part 4/4)

Part 4: Editing-Type Inference and Robustness to Category Misclassification (Q4)

Response: We thank the reviewer for the question. In our original inference procedure, the universal student implicitly identifies the editing type when applying the universal evaluation protocol RuniR_{\mathrm{uni}}, without explicitly outputting a category label. To quantify its category inference accuracy, we modify the output format to additionally require an explicit editing category prediction. The student correctly classifies 670 of 800 samples on PointReward-Bench (83.75%).

Among the 130 misclassified samples, the student's success judgments still agree with human annotations on 110 samples (84.62%), with 45 TP, 9 FP, 65 TN, and 11 FN, using the same fixed criteria (IF ≥9\geq 9, NC ≥8\geq 8, VQ ≥6\geq 6). These results indicate that incorrect category predictions do not necessarily lead to incorrect evaluation judgments. This suggests that the universal student has acquired evaluation capabilities across different editing categories, allowing it to make reliable success/failure judgments based on IF, NC, and VQ even when the explicit category prediction is incorrect.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.