acceptodds
Under review as a conference paper at ICLR 2027

Beyond Random Couplings: Spherical Noise Alignment in Generative Flows

Abstract

Flow matching models are typically trained using independent noise-data couplings, forcing the network to resolve conflicting conditional fields by learning highly curved velocity trajectories. This leads to slow iterative sampling, as numerical ODE solvers require many Neural Function Evaluations (NFEs) to track these paths accurately. While Minibatch Optimal Transport (OT) straightens the vector field by pairing noise and data samples to minimize transport cost, its reliance on discrete matching within small batches leaves it fundamentally bottlenecked by high-dimensional estimation variance. To overcome this limitation, we introduce continuous coupling via noise optimization, a framework that extends beyond discrete index matching towards continuously shaping noise endpoints. As a concrete realization, we present Spherical Noise Alignment (SNA), an online algorithm that rotates source samples through balanced attractive and repulsive forces. The framework aligns noise samples with target data points during training while preserving prior fidelity. This allows seamless sampling from standard Gaussian noise at inference. Experimental results show that our approach improves source-target alignment, significantly reduces path curvature, and achieves superior sample fidelity in few-step (1–4 NFE) sampling regimes.

Primary area:
generative models
Keywords:
Flow Matching, Coupling, Generative Models, Optimal Transport, Contrastive Learning, Diffusion Models, Prior Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Ea26OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Spherical Noise Alignment (SNA), an online continuous noise-data coupling method designed to reduce trajectory curvature in Conditional Flow Matching (CFM). SNA optimizes initial noise directions on the unit hypersphere Sd−1\mathbb{S}^{d-1} via a few projected gradient steps using a contrastive directional alignment loss DS\mathcal{D}_S and a von Mises-Fisher kernel density repulsion regularizer DG\mathcal{D}_G, followed by independent radial resampling ri∼χdr_i \sim \chi_d to preserve the Gaussian prior. Experiments across pixel-space and latent-space image generation benchmarks show that SNA straightens transport trajectories and improves few-step generation quality, particularly when combined with Minibatch OT initialization.

Critical strengths

I found the spherical decomposition of the Gaussian KL divergence in Section 4.2 to be a clean and well-motivated way to tackle batch-level Gaussianization. Enforcing Gaussianity on continuous variables in high dimensions is notoriously tricky, and separating the problem into directional uniformity on Sd−1\mathbb{S}^{d-1}, radial distribution matching, and radius-direction independence in Equation 8 simplifies the design space considerably. By restricting the inner-loop updates to unit vectors and resampling independent radii ri∼χdr_i \sim \chi_d post-hoc, the method eliminates radial mismatch and mutual information by construction. Moreover, connecting the vMF kernel density estimate of angular KL divergence to the log-sum-exp contrastive denominator in Equations 9 through 13 provides a solid theoretical grounding for why repulsive contrastive forces help preserve the prior.

From a practical standpoint, the inner optimization loop is lightweight and easy to integrate into existing flow-matching pipelines. Because the K=4K = 4 updates operate directly on the normalized noise and target vectors without backpropagating through the neural network vθv_\theta, the training throughput overhead is only around 1.5% (Table 8), and inference requires zero extra cost since test-time sampling still draws directly from N(0,I)\mathcal{N}(0, I). I also appreciate the post-hoc path regulation mechanism introduced in Section 5.3 and Appendix C.3. Conditioning the velocity network vθ(xt,t,λS)v_\theta(x_t, t, \lambda_S) on the alignment weight during training acknowledges the inherent trade-off between straight trajectories at low NFEs and exact prior matching at high NFEs, giving practitioners a convenient knob to adjust at inference time without retraining.

Critical weaknesses

My main conceptual reservation is whether continuous directional optimization with a vMF kernel density estimator truly escapes the curse of dimensionality that the authors critique in Minibatch OT. In Section 3.3 and Section 4.1, the paper argues that discrete OT is bottlenecked by the O(N−1/d)\mathcal{O}(N^{-1/d}) empirical convergence rate, whereas SNA bypasses this limit. However, non-parametric kernel density estimation on Sd−1\mathbb{S}^{d-1} in Equations 9 to 12 faces a very similar high-dimensional sample complexity barrier when d=3072d = 3072 or d=4096d = 4096 and the batch size is only N=128N = 128 or 256256.

Specifically, unlike discrete OT, where noise samples are only permuted and every individual sample strictly retains its isotropic Gaussian marginal, SNA actively rotates noise directions u0,iu_{0,i} toward target data directions u1,iu_{1,i}. Since natural images and VAE latents occupy a narrow, anisotropic cone on Sd−1\mathbb{S}^{d-1}, pulling u0,iu_{0,i} toward u1,iu_{1,i} drags the entire noise batch toward the data manifold's angular support. Repelling each noise vector from only N−1N - 1 other batch members (which are themselves being pulled toward the same data cone) prevents pairwise collapse within the batch, but cannot guarantee global uniformity over the entire hypersphere Sd−1\mathbb{S}^{d-1}.

This residual prior drift is visible in the empirical results and even noted in Appendix C.4 (Figure 6), where the authors observe that DG\mathcal{D}_G struggles to Gaussianize 64×6464 \times 64 latents even at N=4096N = 4096. In Table 2, under a 16-step Euler budget on ImageNet32, OT-FM achieves 9.41 FID, whereas standalone SNA degrades to 10.89 and SNA + OT (λS=1/5\lambda_S = 1/5) degrades to 12.82 (requiring λS\lambda_S to be reduced all the way to 1/301/30 to recover 8.96). A similar degradation at 16 steps appears on FFHQ256 (15.73 for OT-FM vs. 17.35 for SNA + OT).

My second concern relates to how standalone SNA performs without Minibatch OT initialization, and what happens to the radial coupling in Algorithm 1. Although the abstract and introduction frame SNA as a continuous alternative that overcomes the limitations and O(N3)\mathcal{O}(N^3) cost of discrete OT, the strongest results in Table 2 rely on SNA + OT, which still runs discrete Minibatch OT first. When looking at standalone SNA (initialized from independent couplings), the gains over OT-FM are marginal or even negative; for instance, on ImageNet32, standalone SNA underperforms OT-FM at 2, 4, 8, and 16 steps (e.g., 105.67 vs. 102.63 at 2 steps, and 39.93 vs. 38.05 at 4 steps).

Questions for authors

I would appreciate it if the authors could address the following questions during the rebuttal:

  1. In Algorithm 1 (line 13), why do you resample new norms ri∼χdr_i \sim \chi_d instead of retaining the original norms ∥x0,i∥2\lVert x_{0,i} \rVert_2 from the initial Gaussian draw? Especially in the SNA + OT setting, doesn't resampling rir_i independently destroy the radial distance matching already established by Minibatch OT? Have you tested keeping the original norms ∥x0,i∥2\lVert x_{0,i} \rVert_2, and how does that affect FID across low and high NFEs?

  2. Since standalone SNA (without OT initialization) underperforms OT-FM on ImageNet32 across 2 to 16 steps and on FFHQ256 at 2 steps in Table 2, why does continuous optimization from random initialization struggle to beat discrete permutation on larger datasets? Is this because K=4K = 4 local gradient steps get trapped in poor local angular assignments without a global combinatorial initialization?

  3. Can you clarify how DG\mathcal{D}_G prevents global angular drift when the target directions u1,iu_{1,i} are concentrated in a non-uniform cone on Sd−1\mathbb{S}^{d-1}? Specifically, have you measured any distributional divergence metric (or cosine similarity distribution against the mean data direction) on the optimized training noise x0∗x_0^* to quantify how much its marginal distribution deviates from N(0,I)\mathcal{N}(0, I) as λS\lambda_S increases?

  4. Since Section 5.1 notes that your experimental setup follows recent coupling works (e.g., Kong et al., 2025; Mousavi-Hosseini et al., 2026; Malnick et al., 2026), could you provide a direct comparison against their reported numbers on CIFAR-10 and ImageNet32 so readers can contextualize SNA's gains relative to contemporary coupling methods beyond vanilla OT-FM?

As a minor note for revision, please fix the notation in Section 5.3, Figure 4, and Table 4 so that the alignment weight is denoted as λS\lambda_S rather than DS\mathcal{D}_S.

5 comments

We sincerely thank the reviewer for the thoughtful and detailed assessment of our work, as well as the interesting points of discussion! We also appreciate the reviewer's acknowledgement of our theoretical contributions and the practical relevance of our method.

Below, we welcome the opportunity to address the specific questions and concerns raised in the review:


Response to Comment 1:

My main conceptual reservation is whether continuous directional optimization with a vMF kernel density estimator truly escapes the curse of dimensionality that the authors critique in Minibatch OT. In Section 3.3 and Section 4.1, the paper argues that discrete OT is bottlenecked by the O(N−1/d)\mathcal{O}(N^{-1/d}) empirical convergence rate, whereas SNA bypasses this limit. However, non-parametric kernel density estimation on Sd−1\mathbb{S}^{d-1} in Equations 9 to 12 faces a very similar high-dimensional sample complexity barrier when d=3072d=3072 or d=4096d=4096 and the batch size is only N=128N=128 or N=256N=256.

We thank the reviewer for this insightful critique. The reviewer is entirely correct: standard non-parametric kernel density estimation in high-dimensions suffers from the curse of dimensionality. However, we want to point out that SNA does not use the vMF kernel to estimate the true, global high-dimensional probability density function. Instead, the log-sum-exp formulation (Eq. 13) serves as an interacting particle system regularizer: specifically, its gradient requires only a local repulsive field rather than an exact, unbiased global density estimate across the entire high-dimensional manifold. This is the same underlying reason why contrastive learning (InfoNCE, Wang & Isola, 2022) works effectively in high dimension. We have added this clarification into the revised manuscript.

Specifically, unlike discrete OT, where noise samples are only permuted and every individual sample strictly retains its isotropic Gaussian marginal, SNA actively rotates noise directions u0,iu_{0, i} toward target data directions. Since natural images and VAE latents occupy a narrow, anisotropic cone on u1,iu_{1, i}, pulling toward drags the entire noise batch toward the data manifold's angular support. Repelling each noise vector from only N−1N-1 other batch members (which are themselves being pulled toward the same data cone) prevents pairwise collapse within the batch, but cannot guarantee global uniformity over the entire hypersphere Sd−1\mathbb{S}^{d-1}.

We thank the reviewer for this sharp observation! The reviewer is entirely right that finite batch sizes NN cannot guarantee global uniformity. This is especially true as we approximate the true KL divergence using a MC estimate with the population limit decaying as O(N−1/2)\mathcal{O}(N^{-1/2}) (Wang & Isola, 2022) in the number of negative samples. This is indeed a valid limitation of our method, and we have included this observation in the revised manuscript (Limitations & Future Work). Nevertheless, DG\mathcal{D}_G still offers a useful heuristic in practice, even for limited batch sizes. This also explains why it is favorable to initialize from noise, which strongly mitigates this limitation. Thank you again for this insightful observation.


Response to Comment 2:

My second concern relates to how standalone SNA performs without Minibatch OT initialization, and what happens to the radial coupling in Algorithm 1. Although the abstract and introduction frame SNA as a continuous alternative that overcomes the limitations and O(N−1/2)\mathcal{O}(N^{-1/2}) cost of discrete OT, the strongest results in Table 2 rely on SNA + OT, which still runs discrete Minibatch OT first. When looking at standalone SNA (initialized from independent couplings), the gains over OT-FM are marginal or even negative; for instance, on ImageNet32, standalone SNA underperforms OT-FM at 2, 4, 8, and 16 steps (e.g., 105.67 vs. 102.63 at 2 steps, and 39.93 vs. 38.05 at 4 steps).

We appreciate the reviewer's concern regarding the relationship between SNA and Minibatch OT. From our perspective, SNA can be viewed through two complementary lenses: (1) Standalone SNA as an efficient alternative to Minibatch OT (in fact, on ImageNet32, our method is 1.34×1.34 \times faster -see Table 11- with roughly similar performance), and (2) SNA + OT as complementary methods, where Minibatch OT acts as an optional initialization step that pre-aligns the pairings, allowing SNA to converge more effectively to a better local minimum and achieve maximum performance. This highlights the practical flexibility of our approach.

Response to Question 1:

In Algorithm 1 (line 13), why do you resample new norms ri∼χdr_i \sim \chi_d instead of retaining the original norms ∥x0,i∥2\left\lVert x_{0,i} \right\rVert_2 from the initial Gaussian draw? Especially in the SNA + OT setting, doesn't resampling rir_i independently destroy the radial distance matching already established by Minibatch OT? Have you tested keeping the original norms ∥x0,i∥2\left\lVert x_{0,i} \right\rVert_2, and how does that affect FID across low and high NFEs?

Thank you for this insightful question! Sampling them independently is a technical choice to fully ensure the independence between the radial and directional distributions by construction. However, we also tested keeping the initial norms ∥x0,i∥2\left\lVert x_{0,i} \right\rVert_2, and the performance was similar (see Table below).

 Euler-2Euler-16
Sampling  ri∼χdr_i \sim \chi_d56.408.23
Restoring original norms ∥x0,i∥2\left\lVert x_{0,i} \right\rVert_257.608.51

We have included this ablation in Appendix C.5. While it's true that independent sampling destroys the radial matching from the OT initialization, our findings indicate that the directional component plays a much more significant role (especially in un-crossing conflicting trajectories). Another intuition is that in high-dimensions the norm of noise vectors cluster tightly in a narrow shell around d\sqrt d, making the effect of radial matching less critical since they tend to share similar magnitudes anyway. Nevertheless, the OT initialization remains useful because OT naturally encourages angular alignment (and reduces directly to it on the unit sphere), thus pre-structuring the initial coupling on Sd−1\mathbb{S}^{d-1}. Although this work focuses primarily on spherical alignment, we agree that exploring the deeper relationship to radial matching is an exciting direction that deserves to be studied in more detail in future work.


Response to Question 2:

Since standalone SNA (without OT initialization) underperforms OT-FM on ImageNet32 across 2 to 16 steps and on FFHQ256 at 2 steps in Table 2, why does continuous optimization from random initialization struggle to beat discrete permutation on larger datasets? Is this because K=4K=4 local gradient steps get trapped in poor local angular assignments without a global combinatorial initialization?

This intuition is completely right, and we agree that this is exactly what happens. The repulsive forces create a complex optimization landscape such that SNA starting from a random initialization easily gets stuck in poor local minima. This is precisely why combining our method with OT initialization works significantly better. We appreciate this sharp observation and have made sure to clarify this in the updated manuscript.


Response to Question 3:

Can you clarify how DG\mathcal{D}_{\text{G}} prevents global angular drift when the target directions are concentrated in a non-uniform cone on Sd−1\mathbb{S}^{d-1}? Specifically, have you measured any distributional divergence metric (or cosine similarity distribution against the mean data direction) on the optimized training noise x0⋆x_0^{\star} to quantify how much its marginal distribution deviates from N(0,I)\mathcal{N}(0, I) as λS\lambda_S increases?

We thank the reviewer for this insightful question regarding the behavior of DG\mathcal{D}_{\text{G}} under non-uniform target distributions. To address this, we refer the reviewer to the newly added Appendix C.6, where we use PCA to assess structure in the optimized noise distribution. This shows that with proper regularization, the biases due to dominant data directions can be kept minimal.

We have additionally clarified the theoretical details and limitations in our Response to Comment 1.

Response to Question 4:

Since Section 5.1 notes that your experimental setup follows recent coupling works (e.g., Kong et al., 2025; Mousavi-Hosseini et al., 2026; Malnick et al., 2026), could you provide a direct comparison against their reported numbers on CIFAR-10 and ImageNet32 so readers can contextualize SNA's gains relative to contemporary coupling methods beyond vanilla OT-FM?

We appreciate this suggestion and agree that it provides valuable context for the reader. In response, we have added a new section (Sec. 5.5: Comparison with Alternative Coupling Methods) in the revised manuscript.

Specifically, we compare SNA with the following baselines:

  • AlignFlow (Kong et al., 2026)
  • SD-FM (Mousavi-Hosseini et al., 2026)
  • MAC (Lin et al., 2025)
  • OT by Design (Malnick et al., 2026)
  • COT-FM (Chiang et al., 2026)

We use reported results across CIFAR-10, ImageNet-32, and ImageNet-256. Additionally, we ran AlignFlow (closely related to SD-FM) on CIFAR-10 using the exact same setup.

Below is the direct comparison (Euler FID-50k ↓\downarrow) by dataset:

1. CIFAR-10 Results

MethodNFEFID
OT by Design (Malnick et al., 2026)1.7287.82
AlignFlow (Kong et al., 2026)269.69
AlignFlow (Kong et al., 2026)425.31
MAC + OT (Lin et al., 2025)430.69
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)256.40
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)422.64

2. ImageNet-32 Results

MethodNFEFID
SD-FM (Mousavi-Hosseini et al., 2026)4~28.3
SNA + OT (Ours, λS=1/5\lambda_S = 1/5)426.86

3. ImageNet-256 Results (SiT-B/4)

MethodNFEFID
OT by Design (Malnick et al., 2026)1.09248.42
COT-FM (Chiang et al., 2026)1241.18
SNA + OT (Ours, λS=1\lambda_S = 1)1245.62

The results show that our method is highly competitive even against these stronger baselines. We have also updated the manuscript to explicitly clarify the conceptual differences between our method and alternatives.


Revision Note:

As a minor note for revision, please fix the notation in Section 5.3, Figure 4, and Table 4 so that the alignment weight is denoted as λS\lambda_S rather than DS\mathcal{D}_{\text{S}}.

Thank you very much for pointing this out! We have fixed this in the revised manuscript.

Reviewer_Bd7xOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how source noise transformations can address the limitations of existing noise-data coupling methods. The authors design a new noise transformation approach while constraining the transformed distribution to remain close to the original Gaussian prior, thereby improving coupling quality and reducing flow trajectory curvature. The authors demonstrate improvements in few-step generation on datasets such as CIFAR-10 and ImageNet.

Critical strengths

Meaningful research problem. The paper investigates noise-data coupling optimization in Flow Matching, which is an important problem for improving sampling efficiency.

Novel perspective and method. The paper provides a new perspective on improving existing coupling methods through continuous noise optimization and proposes a practical approach to achieve this.

Improved Generation Quality. The proposed method demonstrates improvements in generation quality on multiple benchmarks, including CIFAR-10 and ImageNet, particularly in few-step sampling settings.

Critical weaknesses

1. Potential Training-Inference Distribution Mismatch

The proposed method applies a data-dependent transformation to the source noise during training, but still samples directly from the original Gaussian distribution during inference without applying the same transformation. Although the authors introduce a Gaussian regularization term, this regularizer cannot strictly guarantee that the transformed source distribution matches the original Gaussian distribution. Therefore, a distribution mismatch may still exist between training and inference, potentially affecting the effectiveness and stability of the method.

2. Lack of Comparisons with Existing Source Distribution Transformation Methods

Previous works have investigated how source distribution transformations can improve the generation performance of Flow Matching. However, this paper lacks experimental comparisons with these methods. This makes it difficult to assess the actual advantages and effectiveness of the proposed method compared to existing approaches.

3. Insufficient Validation of the Necessity of Continuous Noise Optimization

A core motivation of this paper is that Minibatch OT is limited by the coverage of a finite set of noise samples, which may prevent it from finding suitable matching noise samples for target data. However, if the problem mainly arises from an insufficient number of candidate noise samples, a straightforward alternative would be to increase the size of the candidate noise pool for OT. Although the authors discuss the computational cost of increasing the batch size, they do not provide experimental comparisons between this simple alternative and the proposed method. Therefore, the current experiments are insufficient to fully demonstrate the necessity of continuous noise optimization.

Questions for authors

1. Gaussian Regularization and Training-Inference Consistency

Although Figure 2 demonstrates the effectiveness of the Gaussian regularization term when used alone, could the authors provide quantitative evidence that the transformed noise distribution remains close to the original Gaussian distribution under the full SNA objective, where the Alignment Loss and Gaussian Regularization are jointly optimized?

In particular, how does this distribution shift vary with the alignment strength? Could it explain the degradation in high-NFE generation quality observed under stronger alignment?

2. Experimental Comparisons and Necessity of the Proposed Method

Could the authors provide experimental comparisons with other Source Distribution Transformation methods to further demonstrate the advantages of the proposed approach?

Furthermore, considering that a core motivation of this paper is the limited noise coverage of Minibatch OT, could the authors include experimental results for a simple baseline that increases the OT Matching Batch Size to validate the necessity of continuous noise optimization?

4 comments

We thank the reviewer for recognizing the relevance of our research problem, the novelty of our method, and our empirical gains! We are also grateful for the constructive feedback and additional experiments proposed, which we welcome the opportunity to address below:


Response to Comment 1:

The proposed method applies a data-dependent transformation to the source noise during training, but still samples directly from the original Gaussian distribution during inference without applying the same transformation. Although the authors introduce a Gaussian regularization term, this regularizer cannot strictly guarantee that the transformed source distribution matches the original Gaussian distribution. Therefore, a distribution mismatch may still exist between training and inference, potentially affecting the effectiveness and stability of the method.

While the reviewer is completely right that there is no strict guarantee of Gaussianity even when including the regularizer (which becomes obvious in the extreme case where λS→∞\lambda_S \to \infty), we hypothesize that when λS\lambda_S is carefully tuned, alignment and Gaussianity can successfully coexist. In high dimensions, it is notoriously difficult to truly verify the Gaussianity of a distribution, and we found that one of the most effective ways to validate the Gaussianity is actually through training a diffusion model. This is also supported by our empirical results.


Response to Comment 2:

Previous works have investigated how source distribution transformations can improve the generation performance of Flow Matching. However, this paper lacks experimental comparisons with these methods. This makes it difficult to assess the actual advantages and effectiveness of the proposed method compared to existing approaches.

We appreciate this suggestion and agree that comparing against existing approaches is helpful. To address this, we have added a new section (Sec. 5.5: Comparison with Alternative Coupling Methods) in the revised manuscript.

Specifically, we compare SNA with the following baselines:

  • AlignFlow (Kong et al., 2026)
  • SD-FM (Mousavi-Hosseini et al., 2026)
  • MAC (Lin et al., 2025)
  • OT by Design (Malnick et al., 2026)
  • COT-FM (Chiang et al., 2026)

We use reported results across CIFAR-10, ImageNet-32, and ImageNet-256. Additionally, we ran AlignFlow (closely related to SD-FM) on CIFAR-10 using the exact same setup.

Below is the direct comparison (Euler FID-50k ↓\downarrow) by dataset:

1. CIFAR-10 Results

MethodNFEFID
OT by Design (Malnick et al., 2026)1.7287.82
AlignFlow (Kong et al., 2026)269.69
AlignFlow (Kong et al., 2026)425.31
MAC + OT (Lin et al., 2025)430.69
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)256.40
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)422.64

2. ImageNet-32 Results

MethodNFEFID
SD-FM (Mousavi-Hosseini et al., 2026)4~28.3
SNA + OT (Ours, λS=1/5\lambda_S = 1/5)426.86

3. ImageNet-256 Results (SiT-B/4)

MethodNFEFID
OT by Design (Malnick et al., 2026)1.09248.42
COT-FM (Chiang et al., 2026)1241.18
SNA + OT (Ours, λS=1\lambda_S = 1)1245.62

The results show that our method is highly competitive even against these stronger baselines. We have also updated the manuscript to explicitly clarify the conceptual differences between our method and alternatives.

Response to Comment 3: Validation of the Necessity

A core motivation of this paper is that Minibatch OT is limited by the coverage of a finite set of noise samples, which may prevent it from finding suitable matching noise samples for target data. However, if the problem mainly arises from an insufficient number of candidate noise samples, a straightforward alternative would be to increase the size of the candidate noise pool for OT. Although the authors discuss the computational cost of increasing the batch size, they do not provide experimental comparisons between this simple alternative and the proposed method. Therefore, the current experiments are insufficient to fully demonstrate the necessity of continuous noise optimization.

We thank the reviewer for this suggestion. To directly address this, we have added experimental comparisons in the revised manuscript (Appendix C.4, Effect of Batch Size scaling) where we scale up the Minibatch OT candidate pool size (NN).

B=128B=256B=512
Euler-4: OT-FM30.0027.5926.16
Euler-4: SNA + OT (λS=1/2\lambda_S = 1/2)22.6422.2221.03
Euler-2: OT-FM91.7583.6575.86
Euler-2: SNA + OT (λS=1/2\lambda_S = 1/2)56.4051.0444.27

As these results show, larger OT batch sizes slightly benefit both baselines, but SNA + OT maintains a consistent and substantial advantage at every scale. In particular, a smaller batch size of N=128N=128 with SNA+OT significantly outperforms OT-FM even when scaled up to N=512N=512.

Simply increasing the OT batch size is insufficient for two key reasons:

  1. Computational Cost: (Exact) OT solvers scale cubically (O(N3)\mathcal{O}(N^3)), making large batch sizes computationally infeasible.
  2. Curse of Dimensionality: Discrete sampling suffers severely from the curse of dimensionality (at a rate of O(N−1/d)\mathcal{O}(N^{-1/d})), leaving sparse coverage, and requiring an exponential number of samples (which quickly gets infeasible due to the cubic scaling).

This is also the exact same motivation behind prior, improved coupling approaches (e.g., Mousavi-Hosseini et al., 2026; Kong et al., 2026). Our empirical results demonstrate that SNA offers a much better alternative than simply brute-forcing OT batch size scaling.


Response to Question 1:

Although Figure 2 demonstrates the effectiveness of the Gaussian regularization term when used alone, could the authors provide quantitative evidence that the transformed noise distribution remains close to the original Gaussian distribution under the full SNA objective, where the Alignment Loss and Gaussian Regularization are jointly optimized? In particular, how does this distribution shift vary with the alignment strength? Could it explain the degradation in high-NFE generation quality observed under stronger alignment?

We thank the reviewer for this interesting question. Unfortunately, verifying exact distributional proximity directly in high-dimensional spaces is notoriously challenging due to the limitations of standard divergence metrics in high dimensions. Instead, we find that downstream generation performance (FID) serves as the most reliable indicator of training-inference consistency. Nevertheless, we have included a visualization in Appendix C.6 regarding PCA analysis along dominant directions, which proved most useful to illustrate the effect of the regularizer.

The reviewer's intuition is correct that overly strong alignment can induce a slight distribution shift, which might account for the degradation observed in high-NFE limits. To address this trade-off more fundamentally, we would like to highlight our post-hoc path regularization framework introduced in Section 5.4, which allows us to train a unified model combining the advantages of both worlds: strong low-NFE performance while further improving the high-NFE limit. We believe this highlights the versatility of our method.


Response to Question 2: Additional experimental results

Could the authors provide experimental comparisons with other Source Distribution Transformation methods to further demonstrate the advantages of the proposed approach? Furthermore, considering that a core motivation of this paper is the limited noise coverage of Minibatch OT, could the authors include experimental results for a simple baseline that increases the OT Matching Batch Size to validate the necessity of continuous noise optimization?

We thank the reviewer again for these suggestions! As discussed in the previous comment, we have added an experimental comparison in Section 5.5 and further experimental results regarding the OT Batch Size in Appendix C.4.

Reviewer_Bd7xOpenReview ↗

Response to Authors

Thank you for the detailed response and the additional experiments.

W3 (increasing the OT candidate pool). The new batch-size experiments address this concern well. SNA + OT at B = 128 outperforms OT-FM at B = 512 (Euler-4: 22.64 vs. 26.16; Euler-2: 56.40 vs. 75.86). This shows that the improvement cannot be obtained simply by enlarging the minibatch OT pool, and it supports the necessity of continuous noise optimization. Extending this comparison to a larger dataset (e.g., ImageNet-32) would make the result stronger, but I consider this concern resolved.

W1 (training–inference mismatch). I agree that this is not a fundamental problem for the method, and the trade-off is acknowledged in the Limitations section. Still, I encourage the authors to add simple quantitative diagnostics rather than relying on FID alone. Since the radial distribution matches χ_d by construction, only the directional distribution needs to be checked. For example, the authors could compare the pairwise cosine similarities of the optimized directions with those of fresh Gaussian samples, or report MMD / sliced Wasserstein distances to fresh Gaussian batches across different λ_S. This would make the relationship between alignment strength and high-NFE degradation (e.g., Dopri5 FID in Tables 3 and 4) clearer.

W2 (comparison with other methods). I appreciate the new Sec. 5.5, but the current comparison is still limited and not fully fair:

  • Most of the added baselines (AlignFlow, SD-FM, MAC, COT-FM) are coupling methods. Among the source-distribution methods discussed in the related work, only OT by Design is included. Methods such as MixFlow (Nayal et al., 2026) and the condition-dependent source of Kim et al. (2026) are not compared.
  • Except for AlignFlow on CIFAR-10, the numbers are taken from the original papers, with different NFEs (e.g., 1.72 vs. 2 and 1.09 vs. 1) and possibly different training setups. On ImageNet-256 at 1 NFE, SNA (245.62) is worse than COT-FM (241.18). In addition, the reported 245.62 does not match Table 2 in the paper (244.59 or 254.94); could the authors clarify which configuration is used?

I would like to see more comparisons under unified settings (same backbone, training budget, NFE, and evaluation protocol), especially with source-distribution transformation methods, in the final version.

Summary. The response resolves my concern about the necessity of continuous noise optimization, and I no longer consider the training–inference mismatch a major issue. I have therefore raised my rating to 3. I still encourage the authors to include more controlled comparisons with source-distribution transformation methods in the final version.

Thank you very much for your thoughtful reading of our response, and for helping us improve the submission. We are glad that we could resolve your concerns, and are grateful for your score increase! Also, we thank you very much for your suggestions, and we will make sure to run additional experiments for a more comprehensive comparison in Section 5.5 and include additional metrics regarding W1 in the final version.

Regarding the remaining concerns:


Most of the added baselines (AlignFlow, SD-FM, MAC, COT-FM) are coupling methods. Among the source-distribution methods discussed in the related work, only OT by Design is included. Methods such as MixFlow (Nayal et al., 2026) and the condition-dependent source of Kim et al. (2026) are not compared.

We thank you for this suggestion and agree that additional comparisons to Nayal et al. (2026) and Kim et al. (2026) will be useful for the reader. We will make sure to run additional baselines under unified settings and include the results in the final version.


Except for AlignFlow on CIFAR-10, the numbers are taken from the original papers, with different NFEs (e.g., 1.72 vs. 2 and 1.09 vs. 1) and possibly different training setups.

Thank you for pointing this out. OT by Design (Malnick et al., 2026) uses effective NFEs to account for the inference passes of the auxiliary diffusion model, which is why it is not possible to compare with the exact same NFE. But we completely agree that this makes the comparison tricky.


The reported 245.62 does not match Table 2 in the paper (244.59 or 254.94); could the authors clarify which configuration is used?

We thank you for this sharp observation. The ImageNet 256 result in Table 7 uses SiT-B/4 (to ensure a fair comparison with the other papers), whereas our main results in Table 2 use SiT-B/2. We will make sure to highlight this difference more clearly to avoid any confusion.


Thank you once again for your constructive feedback and for helping us improve the paper!

Reviewer_RUtKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel method to couple the noise and data points in training Flow Matching models. The method uses contrastive loss to rotates the source samples such that the source samples and the data samples are aligned as well as the KL-divergence so that the rotated samples are still Gaussian. The authors conducted experiments on the CIFAR-10 and the ImageNet datasets showing the effectiveness of the proposed method.

Critical strengths

The idea of this paper is novel. Previous methods: Kong et al., 2026; Mousavi-Hosseini et al., 2026, compute the noise-data alignment using the offline semi-discrete optimal transport method. This paper propose to align the noise and data in an online manner. Before feeding a batch of sampled noise and data points to the Flow Matching training model, the method propose to optimize the source points, such that the source points (noise points) and the data points are aligned. To achieve this, the authors use the contrastive loss to align the nose points and data points. To ensure the Gaussianity, the authors propose to decompose the source points into length and direction components and optimize the KL-divergence of the angular term. The authors use a kernel-based density estimation method to estimate the density of the unit components.

Critical weaknesses

In the experiments, the authors only compare with weak baseline OT-FM. Since this paper proposes a novel noise-data coupling method, the authors should compare with stronger baselines: Mousavi-Hosseini et al., 2026; Kong et al., 2026.

In the experiments, the authors also need to train their flow matching using better flow matching methods such as ShortCut model and MeanFlow to show that their proposed method can improve a broader category of flow matching methods.

This paper has some hyper-parameters that play important roles to the method's performance. The kernel width γ\gamma in Eq. 9, the τ\tau in Eq. 14. The authors should investigate the sensitivity of these important parameters. Also for the λS\lambda_S in Eq. 5, the authors need to clarify how this parameter is tuned. Is the λS\lambda_S also sensitive to the final performance?

Questions for authors

To enforce Gaussianity, instead of computing the KL-divergence between the distribution Q and the Gaussian distribution in Eq. 9, can we draw samples from the Gaussian, and compute the optimal transport using empirical samples from Q and the Gaussian to optimize the Gaussianity of the samples?

In Eq. 9, why choose the We use a von Mises–Fisher (vMF) kernel, and not other kernels?

4 comments

We sincerely thank the reviewer for confirming the novelty and improvements! We appreciate the helpful suggestions and welcome the opportunity to add the requested evaluations.

In response to the feedback, we have expanded our evaluation in two directions: we added an experiment combining SNA with MeanFlow and a new section comparing SNA with recent coupling methods.


Response to Comment 1: Comparison with Stronger Baselines

In the experiments, the authors only compare with weak baseline OT-FM. Since this paper proposes a novel noise-data coupling method, the authors should compare with stronger baselines: Mousavi-Hosseini et al., 2026; Kong et al., 2026.

We agree that comparing against recent, stronger baselines is important. To address this, we have added a new section (Sec. 5.5: Comparison with Alternative Coupling Methods) in the revised manuscript.

Specifically, we compare SNA with the following baselines:

  • AlignFlow (Kong et al., 2026)
  • SD-FM (Mousavi-Hosseini et al., 2026)
  • MAC (Lin et al., 2025)
  • OT by Design (Malnick et al., 2026)
  • COT-FM (Chiang et al., 2026)

We use reported results across CIFAR-10, ImageNet-32, and ImageNet-256. Additionally, we ran AlignFlow (closely related to SD-FM) on CIFAR-10 using the exact same setup.

Below is the direct comparison (Euler FID-50k ↓\downarrow) by dataset:

1. CIFAR-10 Results

MethodNFEFID
OT by Design (Malnick et al., 2026)1.7287.82
AlignFlow (Kong et al., 2026)269.69
AlignFlow (Kong et al., 2026)425.31
MAC + OT (Lin et al., 2025)430.69
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)256.40
SNA + OT (Ours, λS=1/2\lambda_S = 1/2)422.64

2. ImageNet-32 Results

MethodNFEFID
SD-FM (Mousavi-Hosseini et al., 2026)4~28.3
SNA + OT (Ours, λS=1/5\lambda_S = 1/5)426.86

3. ImageNet-256 Results (SiT-B/4)

MethodNFEFID
OT by Design (Malnick et al., 2026)1.09248.42
COT-FM (Chiang et al., 2026)1241.18
SNA + OT (Ours, λS=1\lambda_S = 1)1245.62

The results show that our method is highly competitive even against these stronger baselines. We have also updated the manuscript to explicitly clarify the conceptual differences between our method and alternatives.


Response to Comment 2: Compatibility with Broader Flow Matching Methods

In the experiments, the authors also need to train their flow matching using better flow matching methods such as ShortCut model and MeanFlow to show that their proposed method can improve a broader category of flow matching methods.

We appreciate this suggestion to evaluate our coupling on broader flow matching frameworks. To address this, we integrated our noise-data coupling into MeanFlow and added the results in a new section (Sec. 5.3).

Below is the performance comparison on CIFAR-10 using MeanFlow (measured in FID-50k ↓\downarrow):

Configuration1-step2-step
OT-FM (baseline)229.9891.75
OT-FM + MeanFlow8.025.16
SNA + OT (λS=1/2\lambda_S = 1/2) + MeanFlow7.885.12
SNA + OT (λS=1/5\lambda_S = 1/5) + MeanFlow7.915.01

The experiments demonstrate consistent performance improvements when applying our method on top of MeanFlow, validating our method’s broad compatibility across different flow matching methods.


Response to Comment 3: Hyperparameters

This paper has some hyper-parameters that play important roles to the method's performance. The kernel width γ\gamma in Eq. 9, the τ\tau in Eq. 14. The authors should investigate the sensitivity of these important parameters. Also for the λS\lambda_S in Eq. 5, the authors need to clarify how this parameter is tuned. Is the λS\lambda_S also sensitive to the final performance?

We agree that the hyperparameters of the method play an important part. The kernel widths τ\tau and γ\gamma govern the sharpness of the alignment and repulsion. While they could be different in theory, it is recommended to keep those at similar ranges to balance the scale between DS\mathcal{D}_S and DG\mathcal{D}_G. We have clarified these practical guidelines in Appendix B.4. Table 4 and Figure 3 provide a quantitative and qualitative ablation regarding τ\tau.

Overall, the framework is robust to these hyperparameters, and we successfully use a single consistent set across all experiments. λS\lambda_S requires the most care, and Figure 5 provides comprehensive ablations showing its effect on final performance (measured by both curvature and FID).

However, as we demonstrate with the post-hoc path regularization framework in Section 5.4 (Table 5), it is not necessary to tune λS\lambda_S a priori. We believe this makes the method highly practical, with minimal tuning needed.

Response to Question 1:

To enforce Gaussianity, instead of computing the KL-divergence between the distribution Q and the Gaussian distribution in Eq. 9, can we draw samples from the Gaussian, and compute the optimal transport using empirical samples from Q and the Gaussian to optimize the Gaussianity of the samples?

While this is an interesting idea, this will likely not be a sufficiently strong regularizer, as Gaussian regularization turns out to be a highly challenging problem in high dimensions. This also motivated one of our core ideas to simplify the problem by decomposing it into its radial and spherical components. We thank the reviewer again for this interesting discussion point.


Response to Question 2:

In Eq. 9, why choose the von Mises–Fisher (vMF) kernel, and not other kernels?

We thank the reviewer for this question. We mainly choose the vMF kernel due to its theoretical properties and elegance. This way, we obtain a simple solution and perfectly recover the InfoNCE loss, which would not be the case for other kernels. Viewed from a different perspective, because of the vMF choice, maximizing our objective is also equivalent to maximizing the empirical differential entropy of the angular distribution on Sd−1\mathbb{S}^{d-1}. We have added additional explanations in the manuscript.

Reviewer_RUtKOpenReview ↗

Response

I would thank the authors for the response. The authors provided results comparing stronger baselines (AlignFlow, MeanFlow) and showed the superiority of the proposed method. The authors' response regarding the hyper-parameters question is convincing. I will increase the rating.

Thank you for your time and valuable feedback! We are grateful for your score increase and glad that we could address your concerns.

Comments on the paper

1 comment

We thank all reviewers for their detailed and constructive feedback, interesting discussion points, and recognition of our method's relevance and novelty! We are also glad that our responses and additional experiments helped address several of the reviewers' concerns.

The feedback helped us further improve the paper. Most importantly, we have added a comparison with stronger coupling baselines (Section 5.5), further quantitative evaluation using MeanFlow (Section 5.3), additional experiments investigating OT batch-size scaling in Appendix C.4, as well as additional theoretical clarifications and discussions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.