- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an editing pipeline which takes user annotations on the image as condition. The annotations form a canvas instruction that specifies where to edit and what to change. A dataset is constructed in order to enable the training, which covers the common editing types like adding, removal, replacement and so on. Qwen-Image-Edit is adapted to take the canvas as condition, and rubrics-based RL is further applied after the finetuning to improve the editing quality. Results show that the proposed pipeline can take the instructions on canvas as input and correctly perform the edits than general baselines.
Critical strengths
- The pipeline is very clear, and every step is cleanly explained and motivated. The results also show that the model can recognize the instructions on the canvas and perform the edits inside the indicated region.
- The data construction pipeline is reasonable. It also covers the most common types of edits.
Critical weaknesses
- It seems that substantial quality for the evaluation is judged by VLM rubrics score, which appears to be the same reward as what's used in the proposed RL finetuning stage. This might give some fairness concerns for the other baselines which did not finetune using the rubrics score.
- It's less surprising that feeding the canvas-annotated images as input to general backbones like Qwen will cause instruction-retention, since those models are not finetuned with these specific canvas images. I understand that the quantitative numbers are both provided for visual-only setup and text-instructed setup. However, for the visual comparison, what's more interesting and more fair is to show the results where for general backbones, the source image + text instruction are used as input. This can show a fair performance comparison.
- One of the potential use case for on-canvas instruction is multi-edit, where there are multiple areas the users intend to edit, and it's easier for the users to tell the model which instruction is for which area. However, multi-edit results are not shown. Otherwise, I currently could not think of the advantages of having instruction on the image. Since this is part of the main contribution of the paper, I'm open to discussions about this with the authors.
Questions for authors
- Does the RL training stage share the same dataset / training examples as the previous finetuning stage?
- As GPT-5.6-sol is used throughout most of the steps, why is instead GPT-5.4-mini used in the RL stage?
- In dataset construction stage, for object replacement, how is the expanded version of M generated? Does the model have difficulty to handle replacement large size mismatch during inference? Dose the model have difficulty to preserve the background when the target object is much smaller than the original size?
5 comments
W2: Fair visual comparisons with clean source images and text instructions
Thank you for this fair point. We agree that comparing VibeEdit against baselines given clean source images and text instructions is much more informative than visual-only inputs. We have added a side-by-side visual comparison across benchmark cases in Appendix D.2 (Figure 7) of the revised paper—please take a look. As shown in the figure, even with detailed text prompts, the baselines frequently edit the wrong instance among similar objects, misplace moved objects, and alter unrelated background details. These visual results align with Table 1, clearly showing why VibeEdit achieves both a higher VLM rubric score and a higher outside-region PSNR.
W3: Practical advantages of canvas instructions and multi-object editing
Thank you for being open to discussing this. Beyond multi-edit, canvas instructions offer clear practical advantages even for single edits, and VibeEdit also supports multi-edit well:
1. Advantages of on-canvas instructions
- Over pure text prompts: Text describes what to change well, but specifying where in words is often tedious and error-prone. When several objects look alike, users have to write long positional descriptions (~21 words in our benchmark, e.g., counting objects from left to right), yet models still frequently pick the wrong instance. Text is also ill-suited for continuous spatial inputs like exact insertion spots or movement paths. Sketching a quick circle or arrow with a short note removes referential ambiguity, feels natural for artists working on tablets or touchscreens without switching between the image and a prompt box, and helps keep unedited areas intact (32.8 dB outside PSNR vs. 24.6 dB for text baselines).
- Over mask/brush interfaces: Mask-based editors require users to carefully paint object silhouettes, clip out-of-mask effects such as shadows or larger replacements, and cannot express source-to-target links for object movement. Rough canvas marks require no boundary tracing and handle all five edit types in one unified interface. Empirically, VibeEdit also outperforms five mask- and brush-guided editors, please see our responses to Reviewer eZ9A-W1&Q1 and Reviewer WWdX-W1&Q1 for the full breakdown.
2. Multi-edit capability
We strongly agree that multi-edit is a natural fit for canvas instructions, since placing each short note next to its mark directly ties every instruction to its target region. Following your suggestion, we evaluated VibeEdit on one-pass multi-object editing and placed the visual results in Appendix D.3 (Figure 8) of the revised paper—please take a look. As shown there, despite being trained only on single-object pairs, our model directly supports simultaneous multi-object edits (even mixing different operation types) in a single pass.
Q1: Data overlap between SFT and RL
Thank you for asking—we will make this explicit in the revised paper. The SFT and RL stages use completely disjoint datasets: while both follow the same data sources and synthesis pipeline, the 3,520 RL conditions are selected from a separate held-out pool with zero image overlap with the SFT training corpus.
Q2: Choice of GPT-5.4-mini as the RL judge
This comes down to balancing judgment precision with online training speed. Data synthesis and benchmark evaluation run offline, so we can afford higher latency and use GPT-5.6-sol for the most accurate judgments. During RL, however, each binary rubric question is queried in a separate API call, averaging about 10 questions per rollout. With 32 conditions per step across our 32 A100 GPUs and 12 rollouts per condition, every single RL step requires roughly online API calls. Using GPT-5.4-mini cuts response latency substantially so the GPUs do not sit idle waiting for rewards.
Q3 (Part 1): Mask expansion for replacement data generation
After getting the binary object mask from SAM 3 using the selected bounding box, we dilate it outward by 16 pixels (a max filter) before feeding it to FLUX.1 Fill. This margin covers thin edges of the original object so no ghosting is left behind, and gives FLUX.1 Fill a bit of extra room to form the new silhouette and blend the boundary cleanly.
Q3 (Part 2): Large size mismatches and background preservation in object replacement
Thank you for raising this question. Although we did not specifically optimize for large size gaps during training, VibeEdit still handles both small-to-large and large-to-small replacements well across many cases, as shown in Appendix D.4 (Figure 9) of the revised paper. Two factors make this possible:
- Soft spatial guidance at test time: VibeEdit does not apply a hard binary mask during inference; the drawn circle only acts as a soft spatial cue. When replacing a small object with a much larger one, such as turning suitcase into refrigerator, the new object can naturally extend past the original object and circle boundaries with realistic contact and shadows.
- Joint training with removal and addition: A replacement with a large size gap is close to removing the old object and adding a new one at its natural scale. When replacing a large object with a much smaller one, such as swapping a smartphone for a tiny coin or a cushion for a tennis ball, our joint training with object removal helps fill in the newly exposed background cleanly, while object addition guides the model to place the target at a natural size rather than stretching it to fill the old outline.
At the same time, we do observe that extreme size differences are harder and have a lower success rate than replacements between similarly sized objects. Improving robustness on such cross-scale replacements will be a direction for our future work, and we thank you again for bringing up this great question.
W1: Fairness of rubric-based evaluation after RL training
Thank you for this thoughtful point. First, we would like to highlight that even before RL,
VibeEdit-basealready outperforms all baselines in Table 1. Second, because SFT relies on a proxy objective (flow-matching MSE) that does not directly measure instruction following or visual fidelity, RL post-training is intended to align the model with what human users actually care about at test time. Using RL rewards that mirror target evaluation criteria to align with human preference is standard practice in both LLM and diffusion RL literature. For example, in LLMs, DeepSeek-R1[1] optimizes rule-based accuracy rewards evaluated on math and coding benchmarks, and Rubrics as Rewards[2] optimizes LLM-judged checklists evaluated on rubric benchmarks like HealthBench; in visual generation, Flow-GRPO[3], DanceGRPO[4], and DiffusionNFT[5] directly optimize target evaluation metrics such as GenEval, OCR, HPSv2.1, and PickScore. Following this practice, we design both our RL reward and benchmark rubric around edit completion, background preservation, and local quality, pairing a lighter RL judge (GPT-5.4-mini) with a stronger test judge (GPT-5.6-sol) alongside outside-region PSNR (which rises from 27.0 to 32.8 dB).Besides, to give the baselines a completely fair, rubric-free comparison, we conducted a blind user study with 10 human raters over 500 pairwise comparisons, evaluating
VibeEditand all six open-source baselines from Table 1 in their stronger text-instructed setting. Human raters were not shown our rubric questions and judged instruction following and visual quality directly. We report strict win rate, Elo score (ties counted as 0.5), and the win/tie/loss match counts:VibeEditsubstantially outperforms all baselines in human preference (59.5% win rate and 1654.7 Elo), confirming that RL brings genuine visual and instruction-following gains that align with human preference rather than giving an unfair rubric advantage. Furthermore, our human audit in Table 10 confirms that theGPT-5.6-soltest judge closely aligns with real human judgments on VibeEdit outputs.[1]. Guo, Daya, et al. "Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning." arXiv preprint arXiv:2501.12948 (2025).
[2]. Gunjal, Anisha, et al. "Rubrics as rewards: Reinforcement learning beyond verifiable domains." International Conference on Learning Representations. Vol. 2026. 2026.
[3]. Liu, Jie, et al. "Flow-grpo: Training flow matching models via online rl." Advances in neural information processing systems 38 (2026): 40783-40818.
[4]. Xue, Zeyue, et al. "Dancegrpo: Unleashing grpo on visual generation." arXiv preprint arXiv:2505.07818 (2025).
[5]. Zheng, Kaiwen, et al. "Diffusionnft: Online diffusion reinforcement with forward process." International Conference on Learning Representations. Vol. 2026. 2026.