acceptodds
Under review as a conference paper at ICLR 2027

MaskFlow: Precise, Consistent and Seamless Regional Image Editing

Abstract

Regional image editing has attracted considerable attention for its spatial controllability. Although instruction-based and mask-based editing methods can achieve strong semantic alignment, reliable regional control remains challenging, where an edit must be accurately localized and naturally integrated with the preserved context. We propose MaskFlow, a training framework for precise localization, consistent background preservation, and seamless boundary transitions. MaskFlow incorporates the mask into the probability path and flow-matching objective, coordinating generation within the editable region with source preservation outside it. The proposed Soft-Poisson De-seaming module further refines the predicted vector field during both training and sampling to improve the smooth integration of the edited foreground with the preserved background. We also introduce MaskEdit-Benchmark for general scene and infographics editing, where prompts describe the desired edits without localization cues, leaving masks to specify the target regions. Experiments on natural scenes and infographic images demonstrate consistent improvements over competing methods in both quantitative and qualitative evaluations.

Primary area:
generative models
Keywords:
image editing, flow-matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VMGoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a mask-guided framework for regional image editing. It incorporates regional constraints into the probability path and training loss, and uses Poisson blending during sampling to refine boundaries. Experiments cover natural scenes and infographics, with evaluations on two backbone models.

Critical strengths

The method specifically addresses three practical challenges: localization, background preservation, and boundary continuity. Comparisons using the same data and number of training steps, together with consistent gains across two backbones, support the effectiveness of the overall method.

Critical weaknesses

Seamless blending is a central contribution, but the quantitative experiments do not directly evaluate boundary continuity. Figure 6 provides qualitative support, whereas the CLIP and FID scores in Table 2 cannot establish whether textures and lines connect naturally across boundaries. The nearly perfect background preservation scores also partly reflect explicit restoration of the source image. Consequently, the benefits of boundary blending and background restoration remain insufficiently distinguished.

Questions for authors

Which existing results distinguish SPD’s improvement in boundary blending from the effects of simply restoring the background?

1 comment
AuthorsOpenReview ↗

Boundary Continuity Evaluation of SPD

Thanks for the suggestions and questions.

W1&Q1: We apologize for the lack of quantitative results for SPD improvement and we supplement it in paper Section 5.2 Table 3. SPD refines the boundary during training and inference to maintain train-test consistency by solving the mask-aware Poisson equation. Figure 6 provides a qualitative comparison of the finetuned baseline, MaskFlow w/o SPD, and the full model. The comparison between MaskFlow w/o SPD and the full model directly illustrates the incremental effect of SPD on boundary continuity. To evaluate the quantitative results, we supplement BDS[1] metric to evaluate the boundary continuity, which quantifies boundary discontinuity between inner and outer transition bands of the masked region. Here we present the results, where lower BDS value supports better continuity.

ModelBDS$\downarrow$
Gemini0.5179
GPT-Image0.5527
RefineAnything0.7719
RegionE0.5762
SpotEdit0.7820
QwenImage-Inpaint0.9352
Bagel0.7881
HiDream-O10.8005
FLUX.2-dev0.7234
QwenImage-25110.6697
MaskFlow0.4511

The results show that MaskFlow obtains the lowest BDS, which supports the contribution of SPD method. We supplement the results in paper Section 5.2 Table 3 and will claim the contribution more clearly.

References:

[1] Liu, Delong, et al. "Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

Reviewer_BDgDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This manuscript tackles the problem of mask-guided region-based image editing. Their key idea is to use poisson blending between the clean background source image and the predicted clean target image during denoising. For this to work, they further alter the training of a flow-based image generation model (Qwen and Flux 2). For this they noise the background portion of the target image less than the foreground editable region and they further apply poisson blending during training. This mixed noising strategy combined with poisson blending during denoising results in edited images which perfectly preserve the background while applying the intended edits to the masked regions.

Critical strengths

-The idea is simple and effective. Applying poisson blending during denoising to my knowledge is novel and seems to work -The manuscript contains a large gallery of different types of edit, showing the versatility of the proposed method -The provided benchmark is useful for assessing region-aware editing models

Critical weaknesses
  • The training data raises concerns, since both training and benchmarking data comes from the data collected by the authors. Maksing strategy, input images, editing prompts might bias the model and result in higher performance for the proposed model compared to other works which gives this work an unfair disadvantage
  • How sensitive is the method to the mask shape and size? Most results presented in the manusciprt are object level and large masks. How well does the method work on small or irregularly shaped masks?
  • In many region-aware editing works, when suppressing edit to a certain region, the model simply neglects the edit since the noise structure pushes the model to create the edit in another region which the blending supresses (See multidiffusion for example). Does the same happen here? How sensitive is the method wrt the sampled noise? -The methodology section is a bit complicated and confusing with undefined terms such as xSx_S. I think simplifying the explanation helps the clarity of the paper a lot
  • How does the result change when the mask changes in a single example? Does the added object change in size, 3D placcement, etc?
Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Data Fairness, Mask Robustness, and Noise Sensitivity

We thank for the comments and practical suggestions.

W1: The training and benchmarking datasets are disjoint. The source scene images are drawn from the publicly available COCO dataset [1] while the infographics are generated, the arbitrary-shaped masks are obtained using SAM [2] and human experts, and the prompts are generated by a VLM. This model-agnostic data construction is intended to minimize the influence of specific designs for MaskFlow and provide a fair basis for comparison. To further support fairness, we evaluate both MaskFlow and baselines on independent public benchmark Inter-Edit[1] without additional training:

ModelFID$\downarrow$CLIP-C$\uparrow$SSIM-BG$\uparrow$
Gemini8.5330.89260.8114
GPT-Image11.1710.87710.8254
RefineAnything8.9850.86510.9993
RegionE9.5780.85930.8959
SpotEdit10.1260.85510.9003
QwenImage-Inpaint7.6490.89530.9399
Bagel10.7950.85370.9080
HiDream-O115.2790.84730.7551
FLUX.2-dev9.8130.87320.7897
QwenImage-251110.5940.88170.8261
MaskFlow7.1710.91400.9999

The results show that MaskFlow performs stably on independent benchmark. We update the paper Section 5.2 Table 2 and we will clarify this more clearly.

W2: The benchmark consists of lots of arbitrary mask sizes with different mask shapes drawn by users. (1) Mask sizes. We group the samples by mask sizes and report metrics and the gains on each group to show the stability of MaskFlow on mask sizes. Following table shows the relative improvement compared with base model QwenImage-2511:

size proportion[0%, 5%)[5%, 10%)[10%, 20%)[20%, 30%)[30%, 50%)[50%, 100%]
FID$\downarrow$21.52(-50.92%)25.86(-43.27%)34.22(-39.39%)50.35(-36.93%)54.11(-39.16%)81.86(-26.86%)
CLIP-C$\uparrow$0.9298(+6.17%)0.9281(+5.82%)0.9415(+6.33%)0.9415(+5.62%)0.9550(+6.81%)0.9481(+2.92%)

As shown in the table, benefited by supervision from the proposed mask-size-aware loss, our method gives more stability with different mask sizes, and it still works with small masks. We update the results in Supplementary Section G of paper.

(2) Mask shapes. To illustrate the results more intuitively, we infer MaskFlow and baselines on masks that are irregular and are even segmented. The visualizations are updated in paper Section 5.5 and Figure 7(c) for better declaration. For example, models should remove objects in the region where the mask is multiply segmented. The results show that MaskFlow correctly finishes this editing while others fail to edit or only edit partly. This stability comes from the training and the design on SPP, where the training uses position-agnostic prompts to force model to attend to region editing with different mask shapes while SPP guides foreground generation with surrounding structures.

We will provide more clear statements in paper.

W3: Thanks for the concern. (1) Stability. We conduct SPP and generate foreground and background together to ensure structure consistency. Furthermore, benefiting by the mask-aware loss and demonstrative prompts during training, MaskFlow is encouraged to attend more on masked area, showing more stable on different initial noises. We infer MaskFlow on different randomly initial noises to illustrate the results and these visualizations are updated in paper Section 5.5 Figure 7(a). The results show that though the noises change, MaskFlow achieves correct editing on the samples even though there are multiple objects with similar semantics. We clearify this in paper Section 5.5.

(2) Symbol definition. We apologize for making you confused during reading methodology. We will define the symbols more clearly, such as xSx_S is the source image.

W4: Based on the proposed SPP and training strategy, MaskFlow shows strong mask-following capability for different masks, encouraging the generated results to follow the masks. We provide examples in paper Section 5.5 Figure 7(b) to show the features where we change the mask shapes, sizes and positions with the same source image, prompt and random seed. As the results illustrated, the added cat changes its 3D placement to follow the mask shapes and changes in size following the mask size while keeping its completeness. Furthermore, drawing the masks in different places pushes the cats to appear in corresponding places. These results show that the added object can change its sizes and 3D placement following the mask, showing the strong mask-following capability of MaskFlow. We will make more clear statement in paper.

References:

[1] Lin, Tsung-Yi, et al. "Microsoft coco: Common objects in context." European conference on computer vision. Cham: Springer International Publishing, 2014.

[2] Kirillov, Alexander, et al. "Segment anything." 2023 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2023.

Reviewer_BDgDOpenReview ↗

Response to rebuttal

Thanks for the detailed response. I still support accepting this paper.

Reviewer_VcNiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MaskFlow, a framework for regional image editing that addresses inaccurate localization, unintended background changes, and visible boundary seams. It incorporates masks into flow matching through Siamese Probability Paths, providing cleaner background context to guide foreground generation. A mask-adaptive loss strengthens supervision within editable regions, while Soft-Poisson De-seaming refines boundary transitions during training and sampling. The authors also introduce MaskEdit-Benchmark, covering natural scenes and infographics. Experiments on two backbones demonstrate improvements in editing quality and background preservation.

Critical strengths
  • The framework jointly addresses localization, background preservation, and boundary continuity through a coherent mask-aware generation process.

  • Soft-Poisson De-seaming integrates boundary refinement into sampling. Qualitative comparisons show smoother transitions between edited regions and surrounding content.

  • Experiments demonstrate improvements across two backbones and two image domains. Ablations against a baseline trained on the same data support the effectiveness of the proposed components.

Critical weaknesses
  • The reported metrics do not directly measure boundary continuity or text correctness. Boundary-region evaluation, human assessment, and OCR-based metrics would better support the claims, especially for infographic editing.
  • The ablations do not establish whether applying SPD throughout sampling is necessary. Comparisons with final-step-only SPD, conventional Poisson blending, and soft-mask compositing would help isolate its contribution.
  • All quantitative experiments use the proposed benchmark, limiting evidence of generalization. The authors should evaluate on an independent benchmark.
  • The computational cost of SPD is not reported. The default configuration uses 50 Jacobi iterations at each of 50 sampling steps, introducing additional computation whose practical impact remains unclear
Questions for authors

Please see the weakness part

1 comment
AuthorsOpenReview ↗

Additional Metrics, SPD Ablations, Independent Benchmark Results and Computation Cost

Thanks for your constructive suggestions. W1: We report BDS[1] for boundary continuity, human assessment for image quality and OCR accuracy by PaddleOCR[2] for infographics editing. The results are as follows:

ModelBDS$\downarrow$Human assessment$\uparrow$OCR$\uparrow$
Gemini0.517940.9%60.61%
GPT-Image0.552719.3%68.18%
RefineAnything0.771946.3%60.61%
RegionE0.576216.0%43.94%
SpotEdit0.78206.4%50.00%
QwenImage-Inpaint0.935228.3%66.67%
Bagel0.788110.6%43.94%
HiDream-O10.80051.1%66.67%
FLUX.2-dev0.723419.4%77.27%
QwenImage-25110.66978.9%62.12%
MaskFlow0.451171.3%84.85%

(1) Boundary continuity. BDS[1] quantifies boundary discontinuity between inner and outer transition bands, the lower it is, the more continuous the boundary is. As the first column shown, MaskFlow gets better BDS with 0.4511 score. This improvement comes from the design of the SPD which de-seams step-by-step during both training and sampling.

(2) Human assessment. We produced a questionnaire with 18 questions to ask 100 human experts to judge the image quality for all anonymous models and vote for the top 3 models at most for each question. The results show that MaskFlow gets more positive preference with 71.3% of the aggregate top-three preference selections.

(3) OCR accuracy. We calculate OCR accuracy by inferring PaddleOCR on edited images and compare OCR results on ground-truths. MaskFlow shows more accurate character editing in infographics. These additional evaluation metrics further support the design of MaskFlow and we update the paper in Section 5.2, Table 3 and Section F for clearly statements.

W2: To further isolate the contributions, we evaluate variants of SPD and report the boundary quality BDS[1]. Note SPD-L for only conduct SPD at the last step, PB for conventional Poisson blending and SPB for soft-mask composited PB. Both PB and SPB are conducted step-by-step. Here are the results:

PBSPBSPD-LSPD(Ours)
BDS$\downarrow$0.46950.47200.46340.4511

The results show that these variants decrease the BDS while the proposed SPD showing more promising boundary continuity, supporting the effectiveness of step-by-step and soft-mask weighted Poisson blending in space and gradient fields. We update the paper and distinguish them in Section 5.3 and Table 6 more clearly.

W3: We additionally evaluate MaskFlow and baselines on the independent public benchmark Inter-Edit[1] without additional training. We report FID for image quality, CLIP-C for correctness and BDS[1] for boundary continuity. The results are shown in the table:

ModelFID$\downarrow$CLIP-C$\uparrow$BDS$\downarrow$
Gemini8.5330.89260.2476
GPT-Image11.1710.87710.2719
RefineAnything8.9850.86510.3689
RegionE9.5780.85930.4538
SpotEdit10.1260.85510.4606
QwenImage-Inpaint7.6490.89530.2477
Bagel10.7950.85370.4673
HiDream-O115.2790.84730.4366
FLUX.2-dev9.8130.87320.3586
QwenImage-251110.5940.88170.2909
MaskFlow7.1710.91400.1266

As shown in the table, we found MaskFlow trained only on MaskEdit performs better than other baselines on full image quality, editing correctness and boundary quality. These results support the effectiveness of the proposed methods which show promising generalization capability. We supplement the independent benchmark evaluation results in paper Section 5.2 and Table 2 and 3 for more clearly declaration and comparisons.

W4: We time the proposed SPD for computational cost with 50 and 100 iterations at each sampling step. The reported results are timed after 2 warmup loops and averaged by 10 samples.

Iter.SPD time (s)sampling time (s)Proportion of sampling (%)
500.35377.420.46
1000.68377.740.88

These results show that SPD contributes subtle latency during sampling, supporting the practical application availability. More details are described in Supplementary Section D and we will claim this more clearly in paper.

References:

[1] Liu, Delong, et al. "Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

[2] Cui, Cheng, et al. "Paddleocr 3.0 technical report." arXiv preprint arXiv:2507.05595 (2025).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.