- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Spherical Noise Alignment (SNA), an online continuous noise-data coupling method designed to reduce trajectory curvature in Conditional Flow Matching (CFM). SNA optimizes initial noise directions on the unit hypersphere via a few projected gradient steps using a contrastive directional alignment loss and a von Mises-Fisher kernel density repulsion regularizer , followed by independent radial resampling to preserve the Gaussian prior. Experiments across pixel-space and latent-space image generation benchmarks show that SNA straightens transport trajectories and improves few-step generation quality, particularly when combined with Minibatch OT initialization.
Critical strengths
I found the spherical decomposition of the Gaussian KL divergence in Section 4.2 to be a clean and well-motivated way to tackle batch-level Gaussianization. Enforcing Gaussianity on continuous variables in high dimensions is notoriously tricky, and separating the problem into directional uniformity on , radial distribution matching, and radius-direction independence in Equation 8 simplifies the design space considerably. By restricting the inner-loop updates to unit vectors and resampling independent radii post-hoc, the method eliminates radial mismatch and mutual information by construction. Moreover, connecting the vMF kernel density estimate of angular KL divergence to the log-sum-exp contrastive denominator in Equations 9 through 13 provides a solid theoretical grounding for why repulsive contrastive forces help preserve the prior.
From a practical standpoint, the inner optimization loop is lightweight and easy to integrate into existing flow-matching pipelines. Because the updates operate directly on the normalized noise and target vectors without backpropagating through the neural network , the training throughput overhead is only around 1.5% (Table 8), and inference requires zero extra cost since test-time sampling still draws directly from . I also appreciate the post-hoc path regulation mechanism introduced in Section 5.3 and Appendix C.3. Conditioning the velocity network on the alignment weight during training acknowledges the inherent trade-off between straight trajectories at low NFEs and exact prior matching at high NFEs, giving practitioners a convenient knob to adjust at inference time without retraining.
Critical weaknesses
My main conceptual reservation is whether continuous directional optimization with a vMF kernel density estimator truly escapes the curse of dimensionality that the authors critique in Minibatch OT. In Section 3.3 and Section 4.1, the paper argues that discrete OT is bottlenecked by the empirical convergence rate, whereas SNA bypasses this limit. However, non-parametric kernel density estimation on in Equations 9 to 12 faces a very similar high-dimensional sample complexity barrier when or and the batch size is only or .
Specifically, unlike discrete OT, where noise samples are only permuted and every individual sample strictly retains its isotropic Gaussian marginal, SNA actively rotates noise directions toward target data directions . Since natural images and VAE latents occupy a narrow, anisotropic cone on , pulling toward drags the entire noise batch toward the data manifold's angular support. Repelling each noise vector from only other batch members (which are themselves being pulled toward the same data cone) prevents pairwise collapse within the batch, but cannot guarantee global uniformity over the entire hypersphere .
This residual prior drift is visible in the empirical results and even noted in Appendix C.4 (Figure 6), where the authors observe that struggles to Gaussianize latents even at . In Table 2, under a 16-step Euler budget on ImageNet32, OT-FM achieves 9.41 FID, whereas standalone SNA degrades to 10.89 and SNA + OT () degrades to 12.82 (requiring to be reduced all the way to to recover 8.96). A similar degradation at 16 steps appears on FFHQ256 (15.73 for OT-FM vs. 17.35 for SNA + OT).
My second concern relates to how standalone SNA performs without Minibatch OT initialization, and what happens to the radial coupling in Algorithm 1. Although the abstract and introduction frame SNA as a continuous alternative that overcomes the limitations and cost of discrete OT, the strongest results in Table 2 rely on SNA + OT, which still runs discrete Minibatch OT first. When looking at standalone SNA (initialized from independent couplings), the gains over OT-FM are marginal or even negative; for instance, on ImageNet32, standalone SNA underperforms OT-FM at 2, 4, 8, and 16 steps (e.g., 105.67 vs. 102.63 at 2 steps, and 39.93 vs. 38.05 at 4 steps).
Questions for authors
I would appreciate it if the authors could address the following questions during the rebuttal:
-
In Algorithm 1 (line 13), why do you resample new norms instead of retaining the original norms from the initial Gaussian draw? Especially in the SNA + OT setting, doesn't resampling independently destroy the radial distance matching already established by Minibatch OT? Have you tested keeping the original norms , and how does that affect FID across low and high NFEs?
-
Since standalone SNA (without OT initialization) underperforms OT-FM on ImageNet32 across 2 to 16 steps and on FFHQ256 at 2 steps in Table 2, why does continuous optimization from random initialization struggle to beat discrete permutation on larger datasets? Is this because local gradient steps get trapped in poor local angular assignments without a global combinatorial initialization?
-
Can you clarify how prevents global angular drift when the target directions are concentrated in a non-uniform cone on ? Specifically, have you measured any distributional divergence metric (or cosine similarity distribution against the mean data direction) on the optimized training noise to quantify how much its marginal distribution deviates from as increases?
-
Since Section 5.1 notes that your experimental setup follows recent coupling works (e.g., Kong et al., 2025; Mousavi-Hosseini et al., 2026; Malnick et al., 2026), could you provide a direct comparison against their reported numbers on CIFAR-10 and ImageNet32 so readers can contextualize SNA's gains relative to contemporary coupling methods beyond vanilla OT-FM?
As a minor note for revision, please fix the notation in Section 5.3, Figure 4, and Table 4 so that the alignment weight is denoted as rather than .
5 comments
Response to Question 1:
In Algorithm 1 (line 13), why do you resample new norms instead of retaining the original norms from the initial Gaussian draw? Especially in the SNA + OT setting, doesn't resampling independently destroy the radial distance matching already established by Minibatch OT? Have you tested keeping the original norms , and how does that affect FID across low and high NFEs?
Thank you for this insightful question! Sampling them independently is a technical choice to fully ensure the independence between the radial and directional distributions by construction. However, we also tested keeping the initial norms , and the performance was similar (see Table below).
| Euler-2 | Euler-16 | |
|---|---|---|
| Sampling | 56.40 | 8.23 |
| Restoring original norms | 57.60 | 8.51 |
We have included this ablation in Appendix C.5. While it's true that independent sampling destroys the radial matching from the OT initialization, our findings indicate that the directional component plays a much more significant role (especially in un-crossing conflicting trajectories). Another intuition is that in high-dimensions the norm of noise vectors cluster tightly in a narrow shell around , making the effect of radial matching less critical since they tend to share similar magnitudes anyway. Nevertheless, the OT initialization remains useful because OT naturally encourages angular alignment (and reduces directly to it on the unit sphere), thus pre-structuring the initial coupling on . Although this work focuses primarily on spherical alignment, we agree that exploring the deeper relationship to radial matching is an exciting direction that deserves to be studied in more detail in future work.
Response to Question 2:
Since standalone SNA (without OT initialization) underperforms OT-FM on ImageNet32 across 2 to 16 steps and on FFHQ256 at 2 steps in Table 2, why does continuous optimization from random initialization struggle to beat discrete permutation on larger datasets? Is this because local gradient steps get trapped in poor local angular assignments without a global combinatorial initialization?
This intuition is completely right, and we agree that this is exactly what happens. The repulsive forces create a complex optimization landscape such that SNA starting from a random initialization easily gets stuck in poor local minima. This is precisely why combining our method with OT initialization works significantly better. We appreciate this sharp observation and have made sure to clarify this in the updated manuscript.
Response to Question 3:
Can you clarify how prevents global angular drift when the target directions are concentrated in a non-uniform cone on ? Specifically, have you measured any distributional divergence metric (or cosine similarity distribution against the mean data direction) on the optimized training noise to quantify how much its marginal distribution deviates from as increases?
We thank the reviewer for this insightful question regarding the behavior of under non-uniform target distributions. To address this, we refer the reviewer to the newly added Appendix C.6, where we use PCA to assess structure in the optimized noise distribution. This shows that with proper regularization, the biases due to dominant data directions can be kept minimal.
We have additionally clarified the theoretical details and limitations in our Response to Comment 1.
Response to Question 4:
Since Section 5.1 notes that your experimental setup follows recent coupling works (e.g., Kong et al., 2025; Mousavi-Hosseini et al., 2026; Malnick et al., 2026), could you provide a direct comparison against their reported numbers on CIFAR-10 and ImageNet32 so readers can contextualize SNA's gains relative to contemporary coupling methods beyond vanilla OT-FM?
We appreciate this suggestion and agree that it provides valuable context for the reader. In response, we have added a new section (Sec. 5.5: Comparison with Alternative Coupling Methods) in the revised manuscript.
Specifically, we compare SNA with the following baselines:
- AlignFlow (Kong et al., 2026)
- SD-FM (Mousavi-Hosseini et al., 2026)
- MAC (Lin et al., 2025)
- OT by Design (Malnick et al., 2026)
- COT-FM (Chiang et al., 2026)
We use reported results across CIFAR-10, ImageNet-32, and ImageNet-256. Additionally, we ran AlignFlow (closely related to SD-FM) on CIFAR-10 using the exact same setup.
Below is the direct comparison (Euler FID-50k ) by dataset:
1. CIFAR-10 Results
| Method | NFE | FID |
|---|---|---|
| OT by Design (Malnick et al., 2026) | 1.72 | 87.82 |
| AlignFlow (Kong et al., 2026) | 2 | 69.69 |
| AlignFlow (Kong et al., 2026) | 4 | 25.31 |
| MAC + OT (Lin et al., 2025) | 4 | 30.69 |
| SNA + OT (Ours, ) | 2 | 56.40 |
| SNA + OT (Ours, ) | 4 | 22.64 |
2. ImageNet-32 Results
| Method | NFE | FID |
|---|---|---|
| SD-FM (Mousavi-Hosseini et al., 2026) | 4 | ~28.3 |
| SNA + OT (Ours, ) | 4 | 26.86 |
3. ImageNet-256 Results (SiT-B/4)
| Method | NFE | FID |
|---|---|---|
| OT by Design (Malnick et al., 2026) | 1.09 | 248.42 |
| COT-FM (Chiang et al., 2026) | 1 | 241.18 |
| SNA + OT (Ours, ) | 1 | 245.62 |
The results show that our method is highly competitive even against these stronger baselines. We have also updated the manuscript to explicitly clarify the conceptual differences between our method and alternatives.
Revision Note:
As a minor note for revision, please fix the notation in Section 5.3, Figure 4, and Table 4 so that the alignment weight is denoted as rather than .
Thank you very much for pointing this out! We have fixed this in the revised manuscript.
We sincerely thank the reviewer for the thoughtful and detailed assessment of our work, as well as the interesting points of discussion! We also appreciate the reviewer's acknowledgement of our theoretical contributions and the practical relevance of our method.
Below, we welcome the opportunity to address the specific questions and concerns raised in the review:
Response to Comment 1:
We thank the reviewer for this insightful critique. The reviewer is entirely correct: standard non-parametric kernel density estimation in high-dimensions suffers from the curse of dimensionality. However, we want to point out that SNA does not use the vMF kernel to estimate the true, global high-dimensional probability density function. Instead, the log-sum-exp formulation (Eq. 13) serves as an interacting particle system regularizer: specifically, its gradient requires only a local repulsive field rather than an exact, unbiased global density estimate across the entire high-dimensional manifold. This is the same underlying reason why contrastive learning (InfoNCE, Wang & Isola, 2022) works effectively in high dimension. We have added this clarification into the revised manuscript.
We thank the reviewer for this sharp observation! The reviewer is entirely right that finite batch sizes N cannot guarantee global uniformity. This is especially true as we approximate the true KL divergence using a MC estimate with the population limit decaying as O(N−1/2) (Wang & Isola, 2022) in the number of negative samples. This is indeed a valid limitation of our method, and we have included this observation in the revised manuscript (Limitations & Future Work). Nevertheless, DG still offers a useful heuristic in practice, even for limited batch sizes. This also explains why it is favorable to initialize from noise, which strongly mitigates this limitation. Thank you again for this insightful observation.
Response to Comment 2:
We appreciate the reviewer's concern regarding the relationship between SNA and Minibatch OT. From our perspective, SNA can be viewed through two complementary lenses: (1) Standalone SNA as an efficient alternative to Minibatch OT (in fact, on ImageNet32, our method is 1.34× faster -see Table 11- with roughly similar performance), and (2) SNA + OT as complementary methods, where Minibatch OT acts as an optional initialization step that pre-aligns the pairings, allowing SNA to converge more effectively to a better local minimum and achieve maximum performance. This highlights the practical flexibility of our approach.