- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a hierarchical framework for multi-step image editing. A planner trained through checklist-guided self-distillation decomposes abstract instructions, an orchestrator learns tool–region selection from execution rewards, and a distilled verifier selects among candidate edits during inference. To reduce training costs, the method approximates trajectory rewards through independent subtask evaluations and filters low-reward subtasks to refine planning. Experiments examine editing performance, component contributions, and transfer across benchmarks.
Critical strengths
-
A coherent framework for complex editing. The separation of planning, tool selection, and verification provides a practical way to coordinate heterogeneous editors for abstract instructions. Evaluating candidate edits on actual intermediate images is particularly well motivated for handling errors during sequential execution (Sections 3.1–3.4).
-
Execution-grounded supervision with useful ablation evidence. Using observed editing outcomes to supervise tool selection provides a concrete learning signal, while offline subtask evaluation makes supervision more tractable. The fixed-plan ablations in Table 3 provide useful evidence for the benefits of trained orchestration and verifier-guided candidate selection.
Critical weaknesses
-
The training objectives are not sufficiently validated against complete instruction fulfillment. The reward approximations in Section 3.2 rely on evaluating subtasks independently on the original image. However, Section 4.6 examines reward differences without establishing whether tool rankings are preserved on intermediate images. The additive approximation is also not directly validated against final-image rewards. Separately, plan refinement removes low-reward subtasks (Section 3.3), which may omit necessary edits. The higher average reward after filtering (Appendix B.6) is an expected consequence of removing low-scoring samples; it does not demonstrate improved planning or preserved instruction coverage. These gaps leave unclear whether the resulting supervision supports successful completion of the original request.
-
The experiments do not sufficiently separate the benefits of learning from additional inference computation. The full system combines multiple editors, candidate executions, verifier selection, and final refinement. Consequently, comparisons with individual editors do not isolate the contribution of the proposed learning mechanisms. Although Table 3 provides a useful controlled orchestration ablation, the end-to-end advantage over alternatives with matched tool-call budgets and verifier access remains insufficiently established. End-to-end ablations of plan filtering and final refinement are also needed to clarify their contributions. Without these controls, I am not convinced that the reported improvements adequately substantiate the paper’s central methodological claims.
Questions for authors
-
How well do tool–region rankings computed on the original image agree with rankings on actual intermediate images, particularly for dependent edits? Does the summed subtask reward reliably predict the final-image reward?
-
What fraction of subtasks is removed by filtering? Under identical inference settings, how does enabling filtering affect coverage of the original requirements and final instruction satisfaction?
-
With the same tool set, verifier, and total tool-call budget, does learned orchestration still outperform untrained or simpler selection policies? How much of the end-to-end improvement remains when final refinement is disabled?
Rebuttal
Reward Approximations
We thank the reviewer for highlighting the importance of validating our reward approximations against final-image quality and instruction fulfillment. We have provided additional empirical evidence in Appendix D of the updated paper (Figs. 13 and 14), as also summarized in our General Response. Specifically, we demonstrate that (1) trajectories with higher summed stepwise rewards generally achieve higher final-image rewards, supporting the use of additive rewards as a proxy for trajectory quality, and (2) tool preferences evaluated on the original image largely transfer to intermediate images, with tool-ranking agreement well above chance (49.2% top-1, 81.5% top-2, vs. 16.7% and 33.3% at chance) and comparable verifier rewards. We emphasize that these are approximations that do not hold for every editing trajectory, but our empirical findings support their utility in providing meaningful supervision while keeping training computationally tractable.
Compute-Matched Baselines, effect of final refinement, simpler selection policies
We thank the reviewer for highlighting the need to disentangle the benefits of learned orchestration from additional inference computation. To address this, we evaluate sequential single-editor baselines using FLUX.1-Kontext-dev and Qwen-Image-Edit, following the same plan as our method. At each step, we generate five candidate edits (k=5) and use the same verifier to select the best candidate, ensuring matched editing-call budgets. As shown in the table in our General Response, our method outperforms sequential FLUX across all three metrics. Sequential Qwen achieves negligible improvements on instruction following (+0.030), but substantially worse identity preservation (−0.495), with comparable visual quality (−0.010). Together with the controlled ablations in Table 3, these results support the benefits of learned orchestration beyond additional inference computation.
We further compare our learned orchestrator against the pretrained Qwen3-VL-Base model, which relies on its existing capabilities to select editing tools without any task-specific training. Under identical settings (same tool set, same verifier, k=5 candidates per step) (k=5), our learned orchestrator improves instruction following by 12.4 percentage points and visual quality by 0.08 points (on a 5-point scale), as shown in Table 3. This demonstrates that learning from editing outcomes improves tool selection beyond what is possible via a simple orchestration policy.
The final refinement step is a simple instruction guided edit with Qwen-Image-Edit with the prompt “Make it a high-quality image. Reduce noise. Improve contrast and brightness”. For each task, we compute an overall reward by taking the geometric mean of its instruction-following, identity-preservation, and visual-quality scores. Averaged across the 200 tasks, the final refinement had a negligible effect, reducing this reward by only 0.096.
Plan Refinement
Reward-based filtering removes 7.45% of subtasks from the original supervision dataset. Compared with our model, the variant without plan refinement achieves 0.135 lower instruction following, but higher identity preservation and visual quality, using the same Gemini judge across 200 test tasks. It is likely that the unrefined planner proposes edits beyond the available tools’ capabilities, resulting in incomplete or minimal changes. Such under-editing leaves more original content intact and introduces fewer artifacts, yielding higher preservation and quality scores despite weaker instruction fulfillment. This pattern is consistent with a trade-off between executing the requested transformation and preserving the input, rather than uniformly better editing performance. Furthermore, final images are evaluated against the original high-level instruction, not the pruned plan, so dropping necessary edits would lower instruction following. Instead, it rises by 0.135 with pruning, suggesting the pruned subtasks were mainly ones the tools could not execute.