acceptodds
Under review as a conference paper at ICLR 2027

On-Policy Self-Distillation for Multi-Turn Image Editing

25 views

Abstract

Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train–test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a benchmark of 100 ten-turn editing sessions for evaluating long-horizon robustness. Experiments across three editing backbones show that MT-OPSD substantially improves long-horizon editing success and reduces multi-turn collapse while largely preserving single-turn editing quality. All code, checkpoints, and data will be publicly released.

Primary area:
generative models
Keywords:
multi-turn image editing, diffusion model, on policy self distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XfagOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents MT-OPSD, an on-policy self-distillation framework for multi-turn image editing. In multi-turn editing, every round after the first is conditioned on the model's own generated output rather than on a real image. This creates a train–test discrepancy (exposure bias) and causes errors to compound across editing rounds. To address this, the authors train the model directly under these exposure-bias conditions using a dual on-policy strategy. A student model learns both to avoid drift when given an "identity" (no-op) edit instruction and to perform edits correctly from an already-drifted input. The teacher is the same model, conditioned instead on a clean image. The authors also introduce a benchmark for long-horizon multi-turn image editing. They demonstrate empirical improvements over several multi-turn editing baselines across three different editing backbones.

Critical strengths
  1. The paper is well written and easy to follow.
  2. The approach appears novel and generally effective. Several non-trivial design choices, such as gated promotion and on-policy training, are well supported by the ablation studies.
  3. The proposed long-horizon benchmark is a valuable contribution that could benefit future research on multi-turn editing.
  4. The method generally outperforms competing approaches at longer editing horizons, which is the regime the paper specifically targets.
Critical weaknesses
  1. Motivation -- Table 1 suggests that recent proprietary models do not suffer from the stated issue. GPT-Image-2 and Nano Banana Pro both achieve near-perfect scores on the proposed LME-Bench, which may indicate that the problem is already largely solved.

  2. Results on the pre-existing protocols (MSE-Bench and ImgEdit) are not very convincing. On Flux2-klein, there is hardly any improvement over the base model, and on Qwen and FireRed, Emu Edit outperforms or closely matches the proposed method in almost all turns. This makes it unclear whether the gains on LME-Bench carry over to other evaluation settings.

  3. The qualitative results show only a subset of the turns, which makes it difficult to assess prompt adherence and overall edit quality across the full editing sequence. Showing all turns, at least in the supplementary material, would help.

Questions for authors

Please address the weaknesses listed above. In addition, I have the following questions:

  1. The rollouts appear to use a fixed sampling recipe (number of sampling steps, guidance scale) throughout training. Does performance hold under different sampling configurations at test time, or is the method tied to the recipe used during training?

  2. The paper motivates L_id by arguing that asking a single edit to both correct accumulated errors and apply the intended change is too difficult. However, L_edit seems to require exactly this. Because the teacher is conditioned on the clean image, the student must produce a clean edited image from a drifted input, which means performing both restoration and editing. This appears to undercut the motivation for L_id, and the two terms seem to define conflicting targets: should the model edit the image it is given, or the "clean" image? Could you clarify how these two objectives are meant to fit together?

Reviewer_3g76OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the degradation of instruction-based image editing models under multi-turn editing. The authors attribute this degradation to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must condition on their own imperfect outputs at inference time. To address this, they propose MT-OPSD, an on-policy self-distillation framework that: (1) constructs on-policy conditioning states containing only model-induced errors via identity-instruction rollouts; (2) trains with a two-branch objective: an identity branch that targets the rollout state itself to prevent further drift, and an editing branch that transfers clean-conditioned editing behavior to self-generated states via sparse velocity matching; (3) employs an adaptive rollout curriculum and gated teacher promotion based on a VLM judge. The authors also introduce LME-Bench, a benchmark of 100 editing sessions (6 local + 4 global edits per session). Experiments on three backbones (Qwen-Image-Edit-2511, FLUX.2-klein-base, FireRed-Image-Edit) show that MT-OPSD improves SR@10 from 0.03-0.15 to 0.38-0.52 and reduces CR@10 to at most 0.04, while largely preserving single-turn editing quality on ImgEdit.

Critical strengths
  1. The identity-rollout construction isolates model-induced errors from intended semantic changes, enabling on-policy self-distillation from single-turn data alone with no multi-turn annotations, external teacher, or reward model. This converts a complex problem into a practical method that can be transfered to any editing backbone via LoRA. The paper meets acceptance mainly due to its generality and easy adoption, despite some doubts on evaluation methods.
  2. The proposed MT-OPSD improves SR@10 from 0.03-0.15 to 0.38-0.52 and reduces CR@10 to ≤0.04 across three different backbones, with ablations that cleanly attribute each component's contribution. This consistency verifys the paper’s diagnosis: multi-turn collapse is a paradigm-level failure of single-turn training, not a model-specific failure. Paired with LME-Bench as a community resource, this endows the work with enduring value and supports my recommendation.
Critical weaknesses
  1. Evaluations in this paper(key metrics and teacher promotions) use only GPT‑4o/VLM judges, with no human verification. SR/CR for LME-Bench, MSE-Bench scores, and gated teacher promotion all rely on this same set of automated judges. There are no human inter-rater agreement tests, no cross-judge ranking comparisons, and no error bars. This matters because the reported performance improvements may stem from features of the judge models instead of robust visual gains. Therefore, evidences from human evalutions should be supplemented.
  2. Some writing mistakes like Line 226(x~0\tilde{x}_0 denotes the latent of the rollout state I~(k)\tilde{I}^{(k)}) etc.
Questions for authors
  1. If replacing GPT-4o with another mainstream VLM, will the ordering be preserved when compared with Emu Edit/VAE-LFA? This directly affects how much confidence I get from Tables 1 and Table 2.
  2. Why all images of LME-Bench are generated from an AIGC Model(Z-Image-Turbo in this paper)? Natural real photographs are important to certify the practical values of algorithms.
  3. Can you provide a direct comparison with MT-EditFlow? As you mentioned, MT-EditFlow is most closely related to the work.
Reviewer_cBGcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Multi-turn image editing, at least for open-weight models, has a cumulative error issue, where the images start to degrade after multiple edits. The paper proposes two losses to fine-tune a diffusion-based image editing model. One is an identity loss to encourage the rollout to be the same as the previous image after the identity edit instruction. Another is the editing loss, where the model applies a single actual edit to an image after some identity rollout and encourages the edited image to be the same as how an original editing model would edit the clean image. The method is shown to improve multi-turn editing results on multiple open-weight models, while lagging behind the proprietary models.

Critical strengths
  1. The paper is easy to follow, and the proposed method is simple to understand.
  2. Within the paper's setup, the comparison against various baselines and model families is done comprehensively, and this method is shown to be effective amongst open-weight models.
Critical weaknesses
  1. The training was never trained on images with multiple different edit instructions. While I agree that it is a simpler design to correct the drift caused by multiple identity transforms and then encourage an edit instruction to be successful after the drift, wouldn't a training scheme that incorporates actual multi-turn edit instructions be better? I understand that my proposal is challenging to design, but I would encourage the author to discuss this challenge more in the rebuttal and also in the revision. This discussion will motivate the simpler design better in my opinion.
  2. It'd be great to study rollout-length generalization. The method is trained on 10 rollout-length max, and test on 10. It will be good to show, at least qualitatively, how edits will look like after 20, or even 50 edits.
Questions for authors
  1. For your benchmark, is there a way to ensure that the 10 consecutive edits make sense? For example, if there's an image of a cup on the table, and the edit instruction is to remove the cup, then make the cup bigger, then this sequence is invalid.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.