acceptodds
Under review as a conference paper at ICLR 2027

Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generation

Abstract

Current flow matching models learn to transport the source i.i.d. Gaussian noise into the target distribution of natural images, yet this source distribution carries no notion of spatial structure. Images however are fundamentally local since nearby pixels are strongly correlated. By sampling the noise independently, we hypothesize that models are implicitly encouraged to exploit less noisy neighbors as context during training, partially bypassing the need to properly learn the true local structure of images. The source distribution, in other words, works against the inductive bias of the image domain. To ameliorate this design discrepancy, we propose StructFlow, which encodes spatial locality directly into the source by having the pixels within a small region share a common noise component. This structured source produces transport paths that are geometrically aligned with image regions, enabling properties that generic flow matching struggles to provide: fine-grained local editing that naturally respects boundaries, robust structure preservation, and smooth semantic interpolation between images. We show that these benefits also extend to large pre-trained models, demonstrating that StructFlow can even be incorporated through a lightweight post-training phase. Comprehensive experiments on multiple datasets, in unconditional, class- and text-conditioned regimes, using different diffusion transformer architectures confirm that StructFlow not only offers competitive image generation quality, but also significantly improves localized controllable re-synthesis.

Primary area:
generative models
Keywords:
flow matching, diffusion models, structured source distribution, image generation, image editing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RPVrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes StructFlow, which is a flow-matching framework that replaces i.i.d. Gaussian noise with spatially structured noise, in which pixels within local regions share a common noise component to introduce an inductive bias aligned with image locality.

To stabilise training, it introduces spatial anchoring, progressive coherence annealing, and mixed-coherence training strategies. Experiments across unconditional, class-conditional, and text-conditioned generation demonstrate competitive image quality, improved local editing and structure preservation, and the potential to adapt pre trained generative models through lightweight post-training.

Critical strengths

The motivation is intuitive, and the concept is simple. The implementation is also straightforward, essentially extending pixel-space flow matching to the superpixel space.

Critical weaknesses
  1. The practical motivation of the paper is not clear. In the classic pixel-space diffusion image generation model, the spatial correlation of natural images is already captured by the inductive bias of the convolutional U-Net used as the denoising network, rather than encouraging spatial-correlation in the initial noise. Similarly, in latent diffusion, the spatial correlation is also encouraged by the VAE latent token computed from spatially local image patches. Also, since flow matching does not fundamentally require an i.i.d. Gaussian source, in fact any source distribution can be used to train the velocity field, the paper needs to establish why structured spatial-correlated noise offers a meaningful advantage over existing approaches.

  2. Excessive dependence on superpixel segmentation maps. The method relies on superpixel maps, but their availability and generation at inference time, particularly for generic text-to-image generation, are unclear. The demonstrated applications, in fig-4 and fig-5 mainly on structure-preserving fine-grained local editing task, and global structure-preserving image colour/style modification, are limited. If segmentation maps are available, the authors should explain why using them as auxiliary conditioning in a standard U-Net or DiT is less effective than modifying the source noise, and provide direct comparisons to justify the added complexity. For a generic image generation task, using a pre-defined super pixel map as a weak guidance can be restrictive.

Moreover, if a superpixel segmentation map is known a priori, why cannot you simply use it as an auxiliary condition (via attention or concatenation) added to the standard denoising pipeline (either UNet or DIT) ? A substantial ablation study or comparison on this is necessary to demonstrate the value of the method.

Questions for authors
  1. Address the above mentioned weaknesses, especially the suggested ablation study of using a super pixel-map as a condition in the denoising /reverse flow match process, and compare the results.

  2. Can you method be applied to latent space where image patch tokens instead of raw pixels are used ? In this case, how to use the super pixel segmentation map ?

5 comments

We sincerely thank the reviewer for the careful reading and the thoughtful questions. We were glad to hear that the motivation felt intuitive and the method simple and clear. Below, we address each of the reviewer's concerns (W: Weakness, Q: Question).


W1. Practical motivation: why structure the source noise?

We agree that the denoiser already carries a locality bias (convolutions in a U-Net, patch tokens in a latent DiT), and that flow matching does not require an i.i.d. Gaussian source. In fact, this flexibility is exactly what StructFlow builds on.

Our motivation comes from the analysis in Fig. 1 and the Finding in Sec. 3.1. Images, and their VAE latents, are strongly local, yet the source noise has no spatial structure at all: every local patch of noise is independent. When we measure within- vs. across-superpixel similarity along the generation trajectory, the noisy state xtx_t starts with essentially no structure. The predicted velocity, however, already shows a large locality gap at t=0t = 0. In other words, the network's own locality bias is spent manufacturing structure that the source does not provide.

StructFlow asks a simple question: what happens if the source already carries this locality? Doing this naively is unstable, so we introduce the three stabilisation strategies of Sec. 3.3. With them, StructFlow matches standard flow matching in image quality. On top of that, local editing and structure-preserving generation appear out of the box:

  • no ControlNet, no adapter and no extra attention layers;
  • no training on any editing data.

A model trained with standard Gaussian flow matching does not have these abilities. It typically needs extra modules or task-specific data to gain them. That, in short, is the practical value of the idea.


W2. Availability of superpixel maps at training and inference

Training. Superpixels come essentially for free: SLIC runs in milliseconds on the training images we already have. We see this as similar in spirit to using a strong VLM to produce dense captions as a cheap but rich training signal.

Inference. We agree that an exact superpixel map is not available when generating from scratch. Sec. 4.5 and Appendix E study several cheap options:

  • sampling a mask from a precomputed library
  • a small 14M-parameter mask generator (7–35 ms per mask)
  • fully training-free procedural masks such as Poisson-disk Voronoi, whose construction cost is negligible

The model readily accepts these masks and generates high-quality images with them. For structure preservation and local editing, the mask is simply the SLIC map of the image being edited or followed, so it is always available.

Two of our other responses give further evidence on this point. In our response to Reviewer 6GHi (Q2), we quantify how far each inference mask type is from the training SLIC masks and how this affects generation quality. In our response to Reviewer Pqhj (W1), we study what happens when the model is trained on masks that are independent of the image.


W2. Global structure-preserving colour/style modification

Our method does support global structure-preserving modification. Keeping the same structured noise and changing only the prompt changes the colour and style of the whole image while its layout is preserved. This needs no inversion, attention injection or additional control.

Following the reviewer's concern, we conducted additional experiments to support this claim. The new qualitative results are available here: https://anonymous.4open.science/api/repo/StructFlow-ICLR-2027-Rebuttal/file/rebuttal_global_style.png (also Fig. 11 in the revised paper).

W2 and Q1. Why not use the superpixel map as an auxiliary condition?

We thank the reviewer for this suggestion and additional ablation. We agree that it is the most important comparison to make, and we ran it directly.

We would also like to note the broader perspective. Conditioning branches such as ControlNet, channel concatenation or extra cross-attention are the established way to add control. StructFlow explores a different route: it places the spatial inductive bias directly in the source noise, with no conditioning branch at all. As Reviewer 6GHi also noted, this "could offer a new paradigm for improving controllability in generative models". We therefore see the two as complementary directions rather than competing ones. Still, we fully agree that the comparison is informative, and we report it below.

There are two conceptual differences:

  1. No architectural change. StructFlow only changes the source distribution. Channel concatenation changes the input shape, and a cross-attention route adds new modules and parameters.
  2. Editing arises naturally. A condition can tell the network where the regions are. With i.i.d. Gaussian noise, however, redrawing the noise inside one region has no reason to stay inside that region. In StructFlow, the region's noise is tied to the region by construction.

Setup. Starting from the same SANA-1600M checkpoint and the same training data and hyperparameters, we post-trained:

  • (i) Channel concat: standard Gaussian flow matching, with the SLIC map concatenated to the DiT input (zero-initialised projection).
  • (ii) Cross-attention: standard Gaussian flow matching, with the SLIC map injected through an extra trained cross-attention layer in every block (zero-initialised output).
  • (iii) StructFlow (SLIC), post-trained with a comparable short budget, for a fair comparison.

In terms of cost, StructFlow uses exactly the same network as standard flow matching, with no extra parameters or FLOPs. The cross-attention variant adds 201M parameters (+12.5%) and 17% more FLOPs per forward pass. Channel concatenation is nearly free (+0.07M parameters), but it changes the input layer of the model.

Due to the limited rebuttal period, all of them were post-trained for 3k steps.

We then evaluate two tasks, using the same protocol for every model:

  • Structure preservation (PartiPrompts, same setup as in the paper): each model generates from the SLIC map of a reference image, with all noise redrawn per sample. We measure how well the layout of the reference is kept (DINO self-similarity, depth RMSE, Canny-edge F1), sample diversity, and quality.
  • Local editing (PIE-Bench++, same setup as in the paper): each model generates a base image, CLIPSeg selects the region to edit, and only the noise inside that region is redrawn. There is no blending and no masking during denoising. We measure change outside the region (lower is better) and diversity inside it (higher is better).

Structure preservation

MethodSelf-sim ↓Depth RMSE ↓Canny F1 ↑CLIP ↑CLIP-IQA ↑
(i) Channel concat0.2420.2170.51628.270.751
(ii) Cross-attention0.2410.2170.50027.980.742
(iii) StructFlow (SLIC)0.2210.1960.53928.010.746

Local editing

MethodOutside LPIPS ↓Outside RMSE ↓Inside LPIPS ↑CLIP-IQA ↑
(i) Channel concat0.2260.2110.1820.736
(ii) Cross-attention0.2280.2120.1770.738
(iii) StructFlow (SLIC)0.1910.1720.1790.735

All three models can follow the reference layout to some extent, and under the same budget StructFlow already preserves the structure slightly better. The main gap is in local editing. With the auxiliary-condition models, redrawing the noise inside the selected region also noticeably changes the rest of the image. StructFlow keeps the outside of the region clearly better preserved (lower Outside LPIPS and RMSE) while keeping the same diversity inside it (similar Inside LPIPS).

Q2. Does the method apply in latent space, and how is the superpixel map used there?

Yes. In fact, all of our text-to-image results already operate in latent space: SANA is a latent DiT on a DC-AE (f32) latent. VAE latents keep the spatial layout of the image, since each latent position corresponds to a fixed image patch. This is also why cross-attention maps in latent diffusion models line up so well with objects in the image [1].

Using the superpixel map in latent space is therefore straightforward:

  1. Compute SLIC on the full-resolution image.
  2. Sample the structured noise at image resolution with that mask.
  3. Bicubically downsample the noise to the latent grid and rescale it to unit variance.

This keeps segment boundaries smooth even at a high downsampling factor. We will make this procedure more explicit in the revised paper.


We are grateful to the reviewer for these comments and questions which substantially strengthened the paper. We hope the new experiments and clarifications address the concerns, and we would be happy to discuss any further questions.


[1] Hertz, Amir, et al. "Prompt-to-prompt image editing with cross attention control."

Reviewer_PqhjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that in flow matching models, the standard i.i.d. Gaussian source ignores the spatial locality of natural images. The authors propose StructFlow, which replaces this source with a hierarchical Gaussian prior in which latent pixels within the same SLIC superpixel share a common anchor, and they train it with cosine-induced spatial anchoring, progressive coherence annealing, and mixed-coherence training. Experiments cover DiT on unconditional FFHQ and class-conditional ImageNet, and a post-trained SANA for text-to-image generation, with comparisons to standard flow matching and to dedicated structure-preserving and local editing methods.

Critical strengths
  1. StructFlow changes only the source distribution and leaves the architecture, training objective, and sampler unchanged. With progressive coherence annealing, it can be added to a pretrained text-to-image model through post-training.

  2. Local resampling and structure preservation are shown in unconditional, class-conditional, and text-to-image settings, with comparisons to a broad set of dedicated methods.

Critical weaknesses
  1. The source sample depends on the target image. The mask is computed as SLIC(x1), and with cosine-induced spatial anchoring the anchor is a deterministic function of that mask, so StructFlow is conditioned on the target’s superpixel layout rather than relying on a generic locality prior.

2.Generation quality is not shown to be competitive. On FFHQ, FID rises from 22.68 to 25.80 and FID-DINO from 232.4 to 267.3. ImageNet is evaluated only at a classifier-free guidance scale of 4.0, and the mask source and number of samples used for FID are not reported.

  1. The local editing comparison lacks a standard flow matching baseline that resamples noise only in the selected region.

  2. Table 2 does not support the claim that StructFlow performs comparably to dedicated structure-preserving methods. StructFlow has a higher DINO self-similarity distance and depth RMSE than ControlNet-HED, ControlNet-Canny, and SDEdit. It is close only to T2I-Adapter-Canny, which has higher diversity.

Questions for authors
  1. How does StructFlow compare with standard flow matching that receives the same mask as an input channel, and with StructFlow trained on masks independent of the target image?

  2. How does standard flow matching perform when its initial noise is resampled only inside the selected region, evaluated with the background preservation and diversity metrics of Table 3?

9 comments

We sincerely thank the reviewer for the careful and detailed review. The reviewer's questions were very concrete, and we ran new experiments to answer them directly. Below, we address each of the reviewer's concerns (W: Weakness, Q: Question).


W1. The source sample depends on the target image

The reviewer is right that during training the mask is computed from the target, so each source sample is tied to its target's superpixel layout. This is intentional. Flow matching allows any source distribution and any coupling between source and target. StructFlow uses this freedom to tie each region of the noise to a region of the image. This is exactly what makes structure-preserving generation and local editing possible later, without extra modules or editing data.

At inference, no target image is needed. As shown in Sec. 4.5 and Appendix E, the mask can come from a precomputed mask library, a small mask generator, or a training-free procedural method such as Poisson-disk Voronoi.

Why not train on a generic locality prior, such as Voronoi cells, from the start? Such masks are a much weaker signal. A Voronoi cell often covers parts of several objects and does not follow image edges, so there is little for the model to learn about how regions relate to content. We therefore expect structure following not to emerge.

To test this directly, we post-trained SANA with StructFlow on random Voronoi masks that are independent of the target image. Everything else, including the training budget, coherence annealing and data, was the same as for the SLIC model. Both models are then evaluated on structure preservation:

MethodSelf-sim ↓Depth RMSE ↓Canny F1 ↑CLIP ↑CLIP-IQA ↑
StructFlow trained on Voronoi masks0.2680.2350.48027.750.735
StructFlow trained on SLIC masks0.2210.1960.53928.010.746

The Voronoi-trained model is worse on all three structure metrics. Content-aligned masks during training are what teach the model to follow structure, while generic masks remain perfectly usable at inference for plain generation.


W2. Generation quality and evaluation details

Evaluation details. We thank the reviewer for pointing out these missing details and will add them to the paper. For ImageNet, FID is computed on 50k generated samples. The masks are SLIC superpixels with the same setting as in training (50 segments, compactness 50). Both StructFlow and the baseline are evaluated under the same CFG setup on ImageNet.

FFHQ. We acknowledge the FID gap on FFHQ (22.68 → 25.80). We believe it mainly reflects the training regime rather than the method. The face dataset is much smaller than ImageNet (1.28M images). To leave room for the coherence annealing, we trained a 675M-parameter DiT-XL on it for 130k iterations at batch 64, using the same architecture as for ImageNet for consistency. This is many passes over a small dataset with a large model, a regime where FID becomes sensitive to overfitting and less reliable for comparing methods. On the much larger ImageNet, where this does not happen, StructFlow matches or outperforms standard flow matching on FID, sFID, Precision and Recall (Fig. 3). A smaller model and batch size, better matched to the dataset size, avoid this overfitting regime. We kept DiT-XL in the paper only for consistency with ImageNet, and we will include the smaller-model experiment in the camera-ready version.


W3 and Q2. Standard flow matching with noise resampled only inside the region

We thank the reviewer for this very good suggestion. For a model trained with i.i.d. Gaussian noise, nothing guarantees that redrawing the noise inside a region leaves the rest of the image unchanged. We ran the exact experiment the reviewer asked for.

Setup. We use the same PIE-Bench++ items, seeds and sampler as Table 3. Standard SANA generates the base image, which is the same image our StructFlow row was built from. CLIPSeg then selects the region over the same SLIC mask, and only the Gaussian noise inside that region is redrawn, 4 times, with no blending and no masking during denoising.

MethodStructure Dist. ↓BG PSNR ↑BG LPIPS ↓Diversity LPIPS ↑IQA ↑
Standard FM, noise resampled in region0.08910.230.2170.1770.719
StructFlow0.04217.250.1110.1480.731

Standard flow matching changes the image well beyond the selected region. It keeps the background and the overall structure clearly less well (higher Structure Distance and BG LPIPS), and its higher "diversity" partly reflects these changes leaking out of the region. StructFlow keeps the rest of the image intact while still producing diverse edits inside it.

W4. Comparison with dedicated structure-preserving methods (Table 2)

We thank the reviewer for this careful reading, and we agree that our wording should be more precise. ControlNet-HED and ControlNet-Canny do achieve stronger structure adherence than StructFlow. However, they are built specifically for this task and trained on edge maps, and this strong adherence comes at the cost of diversity: their LPIPS diversity is 0.27–0.33, compared with 0.40 for StructFlow. StructFlow needs no task-specific training, data or module. Even so, it achieves the best text alignment (CLIP 28.25), the second-highest diversity and tied-best image quality. It also follows the structure better than T2I-Adapter-Canny (Self-sim 0.181 vs. 0.183, Depth 0.146 vs. 0.153). We will revise the text to say that StructFlow offers a competitive trade-off between structure adherence and diversity, while remaining comparable to some of the methods trained specifically for this task, such as T2I-Adapter-Canny.


Q1. Standard flow matching with the mask as an input channel, and StructFlow trained on target-independent masks

StructFlow trained on masks independent of the target image. This is the Voronoi experiment in W1 above. Training on target-independent masks removes the ability to follow structure.

Standard flow matching with the mask as an input. We would also like to note the broader perspective. Conditioning branches such as ControlNet, channel concatenation or extra cross-attention are the established way to add control. StructFlow explores a different route: it places the spatial inductive bias directly in the source noise, with no conditioning branch at all. As Reviewer 6GHi also noted, this "could offer a new paradigm for improving controllability in generative models". We therefore see the two as complementary directions rather than competing ones. Still, we fully agree that the comparison is informative, and we report it below.

Starting from the same SANA-1600M checkpoint, data and hyperparameters, we post-trained:

  • (i) Channel concat: standard Gaussian flow matching, with the SLIC map concatenated to the DiT input (zero-initialised projection), exactly as the reviewer suggested.
  • (ii) Cross-attention: standard Gaussian flow matching, with the SLIC map injected through an extra trained cross-attention layer in every block (zero-initialised output).
  • (iii) StructFlow (SLIC), with the same short budget.

In terms of cost, StructFlow uses exactly the same network as standard flow matching, with no extra parameters or FLOPs. The cross-attention variant adds 201M parameters (+12.5%) and 17% more FLOPs per forward pass. Channel concatenation is nearly free (+0.07M parameters), but it requires architecture modification for at least the input layer.

Due to the limited rebuttal period, all of them were post-trained for 3k steps. We evaluate them on structure preservation (as in Table 2) and on local editing, where only the noise inside the selected region is redrawn (as in W3).

Structure preservation

MethodSelf-sim ↓Depth RMSE ↓Canny F1 ↑CLIP ↑CLIP-IQA ↑
(i) Channel concat0.2420.2170.51628.270.751
(ii) Cross-attention0.2410.2170.50027.980.742
(iii) StructFlow (SLIC)0.2210.1960.53928.010.746

Local editing

MethodOutside LPIPS ↓Outside RMSE ↓Inside LPIPS ↑CLIP-IQA ↑
(i) Channel concat0.2260.2110.1820.736
(ii) Cross-attention0.2280.2120.1770.738
(iii) StructFlow (SLIC)0.1910.1720.1790.735

All three models can follow the reference layout to some extent, and under the same budget StructFlow already preserves the structure slightly better. The main gap is in local editing. With the input-conditioned models, redrawing the noise inside the selected region also noticeably changes the rest of the image. StructFlow keeps the outside of the region clearly better preserved, with the same diversity inside it.


We are grateful to the reviewer for these questions. They helped us make the paper clearer and more complete. We hope the new experiments and clarifications address the concerns, and we would be happy to discuss any further questions.

We thank you again for your valuable feedback and comments which have helped to strengthen our paper. As the discussion period is ending soon, we would really appreciate if you could let us know if our responses have addressed your concerns. We will be happy to answer any further questions and address any remaining concerns to the best of our abilities in the remaining time!

Reviewer_6GHiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

See below

Critical strengths
  1. This paper takes a novel perspective on noise scheduling by leveraging superpixel segmentation, where pixels within the same superpixel share a common anchor noise, establishing controllable correlations within each region while decoupling noise across different regions. Embedding spatial inductive biases directly into the source noise, without requiring additional conditioning branches, could offer a new paradigm for improving controllability in generative models.

  2. This paper introduces structured spatial masks as priors for local image editing with flow matching and empirically evaluates several alternative mask-generation strategies at inference time, including Voronoi diagrams and mask libraries. This represents a novel empirical exploration of incorporating spatial partitioning constraints into flow matching and offers valuable insights into the development of structured flow-matching methods.

Critical weaknesses
  1. The proposed method relies heavily on SLIC superpixel masks during training. Although alternative mask-generation strategies are introduced at inference time, model performance may depend strongly on the distributional alignment between inference-time masks and the SLIC masks used during training. Moreover, incorporating masks makes the source distribution data-dependent, rather than a fixed, data-independent prior as commonly assumed in standard flow matching. The paper provides no theoretical analysis to establish whether such a data-dependent source distribution can still ensure valid transport from p0p_0 to p1p_1.

  2. The experimental evaluation raises concerns regarding fairness and benchmark completeness. Many competing baselines are specifically trained for image editing, whereas StructFlow relies on the emergent editing capabilities of a pretrained model. The paper does not sufficiently discuss the implications of this difference for a fair comparison. Furthermore, the local editing evaluation is limited to a subset of PIE-Bench, without a comprehensive evaluation on widely recognized standard benchmarks. Finally, the assessment relies exclusively on proxy metrics, such as LPIPS, PSNR, and CLIP, without human evaluations to validate perceptual quality and editing effectiveness.

Questions for authors

1.It is suggested to provide a theoretical analysis establishing whether the mask-induced, data-dependent source distribution can still guarantee valid flow transport from p0p_0 to p1p_1 under the flow-matching ODE framework.

  1. How can the distributional discrepancy between inference-time masks (e.g., Voronoi masks and mask-library masks) and the SLIC superpixel masks used during training be quantified? What is the relationship between this discrepancy and the degradation in editing performance?

  2. It is suggested to design a fairer experimental protocol for comparing StructFlow with task-specific trained image-editing baselines and clarify the rationale behind the current comparison settings

  3. It is recommended to provide a comprehensive evaluation on the full PIE-Bench benchmark and conduct human user studies to validate the effectiveness of the proposed method for local image editing

  4. It is suggeted to further analyze how the structured source distribution affects ODE trajectories, velocity-field learning, and convergence behavior in flow-matching models.

5 comments

We sincerely thank the reviewer for the thoughtful review, and for recognising the novelty of building spatial inductive bias directly into the source noise. The questions were very helpful, and we ran new analyses and experiments to answer them. Below, we address each of the reviewer's concerns (W: Weakness, Q: Question).


W1 and Q1. Is transport from p0p_0 to p1p_1 still valid with a mask-induced, data-dependent source?

Yes, the source distribution is fixed and only the pairing depends on the data. Every source sample is still random (random per-region noise inside each segment). What we mean by "fixed" is the distribution it is drawn from. We follow the paper's convention, where time t=0t=0 is pure noise and t=1t=1 is real data: ε\varepsilon denotes a source (noise) sample, i.e. x0=εx_0 = \varepsilon, x1x_1 a data sample, and p0p_0 and p1p_1 the source and data distributions. Let M=SLIC(x1)M = \mathrm{SLIC}(x_1) be the superpixel mask of a training image, and let q(M)=P(SLIC(x1)=M)q(M) = \mathbb{P}\big(\mathrm{SLIC}(x_1) = M\big), for x1∼p1x_1 \sim p_1, be the distribution of SLIC masks over the dataset. Given a mask, StructFlow draws ε∣M∼N(μM,ΣM)\varepsilon \mid M \sim \mathcal{N}(\mu_M, \Sigma_M) (Sec. 3.2). A training pair is therefore drawn as

x1∼p1,M=SLIC(x1),ε∣x1∼N(μM,ΣM).x_1 \sim p_1, \qquad M = \mathrm{SLIC}(x_1), \qquad \varepsilon \mid x_1 \sim \mathcal{N}(\mu_M, \Sigma_M).

This defines a joint distribution (coupling) π(ε,x1)\pi(\varepsilon, x_1) whose data marginal is p1p_1 and whose source marginal is

p0(ε)=∑Mq(M)⋅N(ε;μM,ΣM).p_0(\varepsilon) = \sum_{M} q(M) \cdot \mathcal{N}(\varepsilon; \mu_M, \Sigma_M).

This is a single Gaussian mixture over superpixel layouts, in the same way that N(0,I)\mathcal{N}(0, I) is a single distribution even though each draw is random. It does not depend on any particular target image. What depends on the data is only which source sample is paired with which image, exactly as in minibatch optimal-transport flow matching [3, 4], where ε∼N(0,I)\varepsilon \sim \mathcal{N}(0, I) but the pairing is chosen using the data.

Proposition. Let (ε,x1)∼π(\varepsilon, x_1) \sim \pi, let xt=(1−t)⋅ε+t⋅x1x_t = (1-t) \cdot \varepsilon + t \cdot x_1 for t∈[0,1]t \in [0, 1], and define v⋆(x,t)=E[x1−ε∣xt=x]v^\star(x, t) = \mathbb{E}\big[x_1 - \varepsilon \mid x_t = x\big]. Then:

  1. the marginal distribution ptp_t of xtx_t satisfies the continuity equation ∂tpt+∇⋅(ptv⋆)=0\partial_t p_t + \nabla \cdot (p_t v^\star) = 0, with pt=0=p0p_{t=0} = p_0 and pt=1=p1p_{t=1} = p_1. Hence, whenever the ODE ddtx(t)=v⋆(x(t),t)\frac{d}{dt} x(t) = v^\star(x(t), t) is well posed, starting it from x(0)∼p0x(0) \sim p_0 gives x(1)∼p1x(1) \sim p_1.
  2. the flow-matching loss E∥vθ(xt,t)−(x1−ε)∥2\mathbb{E} \Vert v_\theta(x_t, t) - (x_1 - \varepsilon) \Vert^2 (with the expectation over tt and (ε,x1)∼π(\varepsilon, x_1) \sim \pi) is minimised exactly by vθ=v⋆v_\theta = v^\star, so the training objective is unchanged.

Proof sketch. Since ddtxt=x1−ε\frac{d}{dt} x_t = x_1 - \varepsilon, for any smooth test function φ\varphi we have

ddtE[φ(xt)]=E[∇φ(xt)⋅(x1−ε)]=E[∇φ(xt)⋅v⋆(xt,t)],\frac{d}{dt} \mathbb{E}\big[\varphi(x_t)\big] = \mathbb{E}\big[\nabla\varphi(x_t) \cdot (x_1 - \varepsilon)\big] = \mathbb{E}\big[\nabla\varphi(x_t) \cdot v^\star(x_t, t)\big],

where the second equality is the tower property of conditional expectation. This is the weak form of the continuity equation. Part 2 holds because the conditional expectation is the minimiser of the mean squared error. Neither step uses independence between ε\varepsilon and x1x_1. This is the marginal-preservation result of rectified flow [1], which holds for any coupling, and it is also the setting of stochastic interpolants with data-dependent couplings [2].

Regularity. The proposition needs the same conditions as standard flow matching, and they hold. For any λς>0\lambda_\varsigma > 0, the secondary noise makes every ΣM\Sigma_M positive definite. So p0p_0, a finite mixture of non-degenerate Gaussians, has a smooth, everywhere-positive density with finite moments, and so does every ptp_t with t<1t < 1, which makes v⋆v^\star well defined. As λς→∞\lambda_\varsigma \to \infty, p0p_0 becomes the standard Gaussian and we recover standard flow matching.

Inference masks. The guarantee is exact whenever inference masks are drawn from q(M)q(M). This is precisely what library sampling does, which is why it performs like the SLIC oracle. For other mask generators (a learned mask model, Voronoi), the inference source p^0\hat p_0 differs from p0p_0. Let p^1\hat p_1 be the distribution obtained by running the same ODE from p^0\hat p_0. If v⋆(⋅,t)v^\star(\cdot, t) is LtL_t-Lipschitz in xx (the standard condition for a well-posed ODE), Grönwall's inequality gives

W2(p^1,p1)≤exp⁡(∫01Ltdt)⋅W2(p^0,p0),W_2(\hat p_1, p_1) \le \exp\Big(\int_0^1 L_t dt\Big) \cdot W_2(\hat p_0, p_0),

where W2W_2 is the 2-Wasserstein distance.

So the error in the generated distribution is controlled by the mismatch in the source, which comes only from the mask distribution. We quantify this mismatch empirically in Q2. We will include a detailed version of this analysis, with full proofs, in the revised paper.

W2 and Q3 and Q4. Fairness of the comparison and evaluation

We would like to clarify one point, because we think it may read the other way around. As the reviewer notes, the editing baselines (SDEdit, Blended Latent Diffusion, inpainting models, InstructPix2Pix, FLUX-Kontext) are explicitly built or trained for editing, while StructFlow is not trained on any editing data and uses no editing module. This setting therefore puts StructFlow at a disadvantage, not the baselines. That StructFlow is still competitive is, in our view, what makes the result interesting.

Our goal in this experiment is not to claim state-of-the-art editing. The main message of the paper is that a different way of modelling the source unlocks capabilities such as local editing and structure preservation for free. We see the editing comparison as a sanity check that these emergent abilities are practically useful. Future work could build dedicated editing methods on top of a StructFlow model, and we expect them to benefit from it. We will make this rationale explicit in the paper.

Why a subset of PIE-Bench. PIE-Bench is designed for instruction-driven editing of real images. It covers ten edit types, many of which are global (style, background, material) or add or remove objects. Our task is different: re-generating a localised region of an image the model generated itself, with the rest kept fixed. We therefore kept the 70 PIE-Bench items whose edit targets a localised object, and used them for every method. We follow PIE-Bench's standard metrics (structure distance and background PSNR/LPIPS).

Human evaluation. We agree that a user study would further strengthen the evaluation. If the reviewer still finds it necessary, we will gladly include a human evaluation of edit quality and background preservation in the camera-ready version.


Q2. Quantifying the discrepancy between inference-time masks and training SLIC masks

We thank the reviewer for this question. Following the analysis in W1, a different mask distribution changes the source only through its spatial correlation structure: how strongly the noise at two positions is correlated is determined by whether they fall in the same region. We therefore measure the discrepancy directly in noise space. For each mask strategy, we compute the mean spatial autocorrelation map C(Δ)C(\Delta) of the source noise on the latent grid, over the set Ω\Omega of non-zero offsets Δ=(Δx,Δy)\Delta = (\Delta_x, \Delta_y) with ∣Δx∣,∣Δy∣≤8\vert\Delta_x\vert, \vert\Delta_y\vert \le 8 (about twice the width of a segment on the 32×3232 \times 32 latent grid, beyond which two positions rarely share a segment). The discrepancy of a strategy XX is the mean absolute difference to SLIC, Dcorr(X)=1∣Ω∣∑Δ∈Ω∣CX(Δ)−CSLIC(Δ)∣D_{\text{corr}}(X) = \frac{1}{\vert\Omega\vert} \sum_{\Delta \in \Omega} \vert C_X(\Delta) - C_{\text{SLIC}}(\Delta) \vert, where CSLICC_{\text{SLIC}} is computed on an independent set of SLIC masks.

We then use the ImageNet StructFlow model (trained only with SLIC masks) to generate with each strategy, and measure generation quality (FID and IS on 5k samples, with the same classes and seeds for all strategies). Due to the limited rebuttal time, we use 5k samples and 50 sampling steps here, instead of the 50k samples and 100 steps used in the paper (Fig. 3, Table 1). The absolute FID and IS values are therefore not comparable to those in the paper, and only the relative comparison across strategies is meaningful.

Inference maskDcorrD_{\text{corr}} (×10³) ↓FID ↓IS ↑
SLIC (library)0.217.9239.6
Perturbed Voronoi1.816.4199.0
Poisson-disk Voronoi2.516.8200.9
Regular grid5.222.3128.0
Smooth deformation6.219.5145.8
i.i.d. Gaussian (no mask)631318.22.1

The discrepancy orders the strategies exactly as the qualitative results in Appendix E: SLIC < Voronoi variants < grid and smooth deformation ≪ i.i.d. noise. It closely tracks generation quality (Pearson r=−0.95r = -0.95 with IS across the five mask strategies). The Voronoi masks, which are closest to SLIC, lose little: FID is even slightly lower, and IS is lower. This is consistent with the bound in W1, where the degradation grows with the mismatch of the source.


Q2. Why not train with Voronoi masks directly?

A Voronoi cell often covers parts of several objects and does not follow image edges, so it is a much weaker signal about how regions relate to content. We tested this by post-training SANA with StructFlow on random Voronoi masks, with everything else the same as for the SLIC model:

MethodSelf-sim ↓Depth RMSE ↓Canny F1 ↑CLIP ↑CLIP-IQA ↑
StructFlow trained on Voronoi masks0.2680.2350.48027.750.735
StructFlow trained on SLIC masks0.2210.1960.53928.010.746

The Voronoi-trained model follows structure clearly worse. Voronoi masks remain a good, nearly free choice at inference for plain generation, but content-aligned masks during training are what teach the model to follow structure.


Q5. Effect on ODE trajectories, velocity-field learning and convergence

ODE trajectories. The minimum of the flow-matching loss, attained at v⋆v^\star, is E∥(x1−ε)−v⋆(xt,t)∥2\mathbb{E} \Vert (x_1 - \varepsilon) - v^\star(x_t, t) \Vert^2, i.e. the expected conditional variance of x1−εx_1 - \varepsilon given xtx_t. It is zero exactly when x1−εx_1 - \varepsilon is fully determined by xtx_t, that is, when no two source–target paths cross. In that case the ODE trajectories are straight lines [1]. With an i.i.d. source, ε\varepsilon carries no information about x1x_1, so many paths cross and v⋆v^\star must average over them. In StructFlow, the source already carries the layout of its target, which makes the target velocity less ambiguous given xtx_t.

To test this, we measured the ODE trajectories of base SANA and StructFlow-SANA on the 64 PartiPrompts (2 seeds each, CFG 4.5, the same 50-step Euler sampler for both), with StructFlow using the SLIC mask of a reference image, as in the structure-preservation setup. For a sampled trajectory with initial noise x0=εx_0 = \varepsilon, final sample x1x_1 and model velocity vt=vθ(xt,t)v_t = v_\theta(x_t, t), we report the straightness S=1∥x1−x0∥2∫01∥vt−(x1−x0)∥2dtS = \frac{1}{\Vert x_1 - x_0 \Vert^2} \int_0^1 \Vert v_t - (x_1 - x_0) \Vert^2 dt (0 means a perfectly straight path) and the cosine similarity between vtv_t and x1−x0x_1 - x_0, averaged over the sampling steps. Both are averaged over all samples. We also report how close 4-, 8- and 16-step samples are to the 50-step sample from the same noise.

MethodStraightness SS ↓Cosine ↑LPIPS to 50-step, 4 / 8 / 16 steps ↓
Standard FM (base SANA)0.2630.8780.485 / 0.294 / 0.138
StructFlow0.1980.9030.243 / 0.126 / 0.051

StructFlow's trajectories are about 25% straighter, and a 4-step sample is already much closer to the final image (LPIPS 0.24 vs. 0.49). This matches the analysis above: because the source carries the layout of its target, the velocity is less ambiguous and the path from noise to image is more direct.

Velocity-field learning. Fig. 1 of the paper shows that with an i.i.d. source, the velocity field already has a large within- vs. across-superpixel similarity gap at t=0t = 0, while xtx_t has none. In other words, the network spends capacity manufacturing locality that the source does not provide. With StructFlow, this locality is already in the source. Consistently, Appendix C.3 shows that coherent object boundaries appear in the DiT features after only 10% of the trajectory, compared with 50–90% for standard flow matching.

Convergence. The three stabilisation strategies in Sec. 3.3 address convergence directly. Naive structured noise trains poorly (FID 18.12 in Table 1 of the paper). Progressive Coherence Annealing starts close to the Gaussian source and gradually adds structure, which makes training stable and lets us post-train pretrained models such as SANA. With the full recipe, StructFlow reaches the quality of standard flow matching under the same training budget (Fig. 3 of the paper).


We are grateful to the reviewer for these questions, which helped us make the paper more rigorous and complete. We hope the new analysis and experiments address the concerns, and we would be happy to discuss any further questions.


[1] Liu, Xingchao, Chengyue Gong, and Qiang Liu. "Flow straight and fast: Learning to generate and transfer data with rectified flow."

[2] Albergo, Michael S., et al. "Stochastic interpolants with data-dependent couplings."

[3] Tong, Alexander, et al. "Improving and generalizing flow-based generative models with minibatch optimal transport."

[4] Pooladian, Aram-Alexandre, et al. "Multisample flow matching: Straightening flows with minibatch couplings."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.