- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper argues that image editing reward models suffer from criterion entanglement because different editing categories require different evaluation and preservation criteria. The authors train specialized reward experts for eight editing categories and consolidate their capabilities into a compact universal evaluator through privileged on-policy distillation. The method is evaluated on pointwise evaluation, preference ranking, data filtering, and downstream RL.
Critical strengths
The paper presents a complete and practically motivated reward modeling framework. It covers category-specific expert training, privileged on-policy distillation, preference adaptation, and downstream RL in a unified pipeline. The PointReward-Bench and downstream RL experiments suggest that the method improves both static reward evaluation and actual editing performance.
Critical weaknesses
-
The necessity of training separate experts for different editing categories is not sufficiently established. Unlike domains such as mathematics and coding, the eight image editing categories share many fundamental capabilities, including instruction following, visual quality assessment, and preservation of unedited regions. It is unclear whether different editing categories truly require independent experts. Category conditioning, category-specific prompts, adapters, or a sufficiently expressive single model might already capture these distinctions. Therefore, it remains unclear whether the gains come from expert specialization itself, or from additional parameters, different data distributions, and more detailed annotation protocols.
-
The reliability of category routing is not sufficiently analyzed. Multi-step edits and edits involving multiple objects may belong to several categories, but the paper does not study ambiguous categories, routing errors, or mixed-category examples.
Questions for authors
See Critical Weaknesses
4 comments
Response to Reviewer ztsV (Part 2/2)
Part 2: Category Routing and Mixed-Category Edits (W2)
Response: We thank the reviewer for the comment. We clarify how MoRE-OPD handles category routing during training and inference, as well as ambiguous and mixed-category editing instructions. To address the concern that category ambiguity may lead to incorrect evaluation judgments, we analyze the impact of category misclassification on success/failure decisions. We have also conducted a category-wise analysis to assess the effectiveness of MoRE-OPD across different editing tasks, particularly multi-step edits. We have added the category-wise results to Appendix E of the revised manuscript.
Reliability of Category Routing. During OPD, we route samples to specialized teachers using dataset-provided category labels, without requiring a separate category classifier. At inference, the universal 9B student implicitly identifies editing types under the universal evaluation protocol , without explicit expert routing or adapter selection.
To examine the relationship between category misclassification and evaluation accuracy, we modify the inference output format to additionally require the student to predict one of the eight editing categories alongside its original evaluation scores and success/failure judgment, without retraining. The student correctly identifies 670 of 800 categories (83.75%). Among the 130 misclassified samples, 110 success/failure judgments remain correct (84.62%). These results indicate that incorrect category predictions do not necessarily lead to incorrect evaluation judgments. This suggests that the universal student has acquired evaluation capabilities across different editing categories, enabling it to assess IF, NC, and VQ even when the explicit category prediction is incorrect.
Handling Mixed-Category Edits. We assign instructions involving multiple editing operations to the Multi-Step Edits category. Its specialized expert evaluates each requested sub-task and the consistency across edits. Importantly, multiple objects do not necessarily imply multiple editing categories: removing two objects remains a removal task, whereas removing an object and modifying text constitutes a multi-step edit.
Our category-wise analysis on PointReward-Bench (Table D-2) shows that MoRE-OPD improves accuracy over the Initial Student in five of the eight editing categories, maintains the same accuracy in two, and decreases in one. Notably, on the 100-sample Multi-Step Edits subset, accuracy improves from 82% to 89%, representing the largest gain across all categories. These results support the effectiveness of our framework across diverse editing tasks, particularly compound editing instructions.
Table D-2. Category-wise evaluation on PointReward-Bench. Each category contains 100 samples. denotes the accuracy difference in percentage points.
| Editing Category | Initial | Student | MoRE-OPD (Ours) | (pp) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Acc. (%) | TP | FP | TN | FN | Acc. (%) | TP | FP | TN | FN | ||
| Add Object or Element | 85 | 43 | 12 | 42 | 3 | 90 | 44 | 8 | 46 | 2 | +5 |
| Alter Background, Texture, or Style | 76 | 33 | 6 | 43 | 18 | 72 | 27 | 4 | 45 | 24 | -4 |
| Change Camera View or Perspective | 76 | 25 | 3 | 51 | 21 | 76 | 25 | 3 | 51 | 21 | 0 |
| Edit People, Faces, or Character Attributes | 82 | 50 | 15 | 32 | 3 | 82 | 44 | 9 | 38 | 9 | 0 |
| Edit Text or Document Content | 85 | 39 | 13 | 46 | 2 | 87 | 35 | 7 | 52 | 6 | +2 |
| Modify Existing Object | 78 | 40 | 13 | 38 | 9 | 84 | 39 | 6 | 45 | 10 | +6 |
| Multi-Step Instruction (Compound Edits) | 82 | 38 | 14 | 44 | 4 | 89 | 36 | 5 | 53 | 6 | +7 |
| Remove Object or Element | 79 | 51 | 15 | 28 | 6 | 81 | 45 | 7 | 36 | 12 | +2 |
| Overall | 80.38 | 319 | 91 | 324 | 66 | 82.62 | 295 | 49 | 366 | 90 | +2.25 |
Review
Thank you for the thorough ablations. Scaling from 1 to 8 LoRA teachers under a fixed backbone and data pool (Table D-1) and the routing/multi-step analysis (Table D-2) directly address my concerns on expert specialization and category ambiguity. Please include these results in the revision. I am raising my Soundness score and updating my rating to 3 (Weak acceptance).
We sincerely thank you for your positive feedback and constructive suggestions. We are pleased that our additional experiments have addressed your concerns. We have incorporated the expert specialization ablations (Table D-1) into Section 4.5 (Table 2), and the category-wise evaluation results, including multi-step edits (Table D-2), into Appendix E of the revised manuscript. All additions and revisions have been highlighted in red in the revised manuscript.
Response to Reviewer ztsV (Part 1/2)
We thank the reviewer for recognizing the practical significance of our work and the effectiveness of MoRE-OPD. We have significantly improved and revised the manuscript and conducted additional experiments to address concerns regarding expert specialization, category-aware routing, and ambiguous or mixed-category editing tasks, as detailed below.
Part 1: Expert Specialization and Category-Specific Adapters (W1)
Response: We thank the reviewer for the comment. We have conducted controlled comparisons between a single category-conditioned teacher and multiple specialized teachers, along with ablations on the number of experts, to assess the necessity of expert specialization. We also clarify the role of category-specific LoRA adapters in our framework. We have added these experiments to Table 2 and Section 4.5 of the revised manuscript, as summarized in Table D-1.
Table D-1. Ablation study on PointReward-Bench, corresponding to Table 2 in Section 4.5 (Ablation Studies) of the revised manuscript. Teachers (Qwen3.6-27B) use category-specific protocols Rc and are evaluated without privileged rationales. All OPD students (Qwen3.5-9B) start from the same SFT checkpoint and use the universal protocol Runi during training and evaluation.
Effectiveness of Multi-Teacher Specialization. To distinguish the contribution of expert specialization from that of category conditioning and category-specific prompts, we compare configurations with 1, 2, 4, and 8 teachers. All configurations use the same Qwen3.6-27B backbone, overall data pool, and category-specific protocols Rc. The single teacher handles all eight categories with their corresponding evaluation criteria, while multiple teachers specialize in assigned categories. For fewer teachers, we group semantically similar categories: four teachers each handle two categories, and two teachers each handle four categories. All configurations supervise the same initial 9B student through identical privileged OPD with the same oracle rationales.
As the number of teachers increases from 1 to 2, 4, and 8, teacher accuracy improves from 77.38% to 79.50%, 79.88%, and 82.12%, while student accuracy increases from 80.12% to 80.50%, 81.50%, and 82.62%. In particular, eight specialized teachers improve student accuracy by 2.50 percentage points over the single-teacher control. These results indicate that a single 27B teacher, even with explicit category conditioning, detailed prompts, and the same privileged rationales, is insufficient to capture category-dependent evaluation distinctions as effectively as multiple specialized teachers. It supports the necessity of multiple experts for providing the stronger supervision and improved distillation performance.
Category-Specific Adapters. We construct our specialized teachers using separate LoRA adapters on the same pretrained Qwen3.6-27B backbone. Each adapter is independently trained on its assigned editing categories with the corresponding evaluation protocols Rc. Thus, category-specific adapters are already part of our expert construction rather than an alternative to it.
However, using these adapters directly at inference would require category-specific adapter selection. Our privileged OPD instead consolidates their specialized evaluation capabilities into a single 9B student under the universal protocol Runi, eliminating the need for multiple experts or adapter switching at inference.