- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
- ControlEdit adapts a text-to-image model without direct supervision from edited target images. Reconstruction-trained ControlNet features provide source conditioning, followed by instruction distillation and reinforcement learning with adaptive conditioning strength.
Critical strengths
- The method offers a practical way to reduce dependence on target editing images.
- Comparisons with supervised models using the same architecture are useful, and component ablations support combining adaptive conditioning with LoRA updates.
Critical weaknesses
- Baseline fairness is not fully established: most models use ten sampling steps, while the supervised counterpart's training budget is insufficiently specified.
- Avoiding target-image supervision does not by itself establish lower overall cost. The method uses an existing paired dataset and substantial RL sampling.
Questions for authors
- Clarification: How are the target descriptions used during instruction distillation obtained? Does their construction require access to edited target images?
- Evidence request: What are the supervised baseline's training budget and tuning procedure? Do the main comparisons hold when competing models use their recommended sampling configurations?
2 comments
Solved my concerns, so I will keep my rating.
We thank reviewer MpaJ for valuable feedback. We address the reviewer's concerns below:
Supervised training budget: Our supervised model uses the same configuration as ControlEdit, including the same ControlNet architecture and a LoRA rank of 128, but without the gating network. The supervised model is trained on the same GPT Image 1.5 M dataset, using ground-truth edited images as supervision. We train the supervised model with a batch size of 8 per GPU on 4 A100 GPUs for 40k iterations, requiring 118 GPU hours.
Sampling configuration of baseline models: The Step1X-Edit and Qwen-Image-Edit baselines use their respective default numbers of sampling steps, 28 and 50. The full results are presented below:
As the number of sampling steps increases, both Step1X-Edit and Qwen-Image-Edit show consistent improvements in editing performance. Our method underperforms these methods, both with the same number of sampling steps and under their default configurations. In our paper, we only claim that ControlEdit outperforms NP Edit in terms of SC and PQ scores.
We will add the refined Table to our revision of this paper.
Target description concerns: ControlEdit does not use target images at any stage. It only requires the source image, editing instruction, and source/target prompts, none of which require access to the edited image. The editing instructions and target prompts can be generated by a language model from the source prompt or source image alone, as demonstrated by InstructPix2Pix, which generates both the editing instruction and edited caption from the input caption before any image synthesis, and NP Edit, which uses a VLM to generate editing instructions and edited-image captions directly from reference images. Therefore, our training data can be constructed from any image corpus without requiring the generation, filtering, or verification of edited images.
Cost for ControlEdit and comparison with the cost of data pipeline: We agree with the reviewer that RL post-training is a computationally expensive stage. Our DiffusionNFT training requires 1,038 GPU hours, while ControlNet pretraining and instruction distillation require 7 and 12 GPU hours, respectively. However, we argue that this cost is still lower than that of an edited-image generation pipeline. To illustrate this, we consider a simple strategy using Qwen-Image-Edit with 50 sampling steps as the data generator. Generating a 512×512 image with Qwen-Image-Edit takes 28 seconds on 2 A100 40GB GPUs, corresponding to approximately 56 GPU-seconds per image. Our training consists of 30k iterations, covering approximately 480k images. Generating the same number of edited images using Qwen-Image-Edit would require more than 7,000 GPU hours, which is substantially more expensive, even without accounting for the verification stage.