acceptodds
Withdrawn from ICLR 2027

Stochastic Sampling with 1D Brownian motion for Flow Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Flow Matching (FM) models are commonly sampled with deterministic ordinary differential equations. Stochastic sampling can improve trajectory diversity and sample quality, but conventional stochastic differential equation (SDE) samplers often require a large number of function evaluations (NFE). Building on the classi- cal marginal-preservation identity for the Fokker–Planck equation, we develop an adapted low-rank step-process construction that converts low-dimensional Brown- ian motion into a practical, training-free sampler. The Simplified 1D-Noise-Guided Sampling (SONGS) framework, in its rank-one form, injects a scalar Brownian increment along a history-dependent estimate of the local discretization-error direc- tion. Freezing this direction on each integration interval eliminates the Jacobian and divergence evaluations required by a general state-dependent diffusion and supports an improved small-noise stochastic Runge–Kutta implementation. Experiments on CIFAR-10, FFHQ, and ImageNet demonstrate that SONGS achieves superior sampling performance at low NFE.

Primary area:
generative models
Keywords:
Flow Matching, Stochastic Differential Equation, Sampling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sgHqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

SONGS proposes a training-free stochastic sampler based on scalar Brownian motion. It estimates a direction from completed solver stages, holds it fixed over the next interval, and injects noise with a drift correction based on the projected score. Experiments show substantial improvements at very low NFE budgets. However, the paper's claim that the sampler preserves marginal distributions for arbitrary adapted step-process directions raises a significant theoretical concern.

Critical strengths

The paper tackles the important practical challenge of generating high-quality samples with very few model evaluations. Its simple, training-free approach and promising experimental results make it a worthwhile direction to explore.

Critical weaknesses
  1. The paper's theoretical analysis does not establish its central distribution-preservation claim as stated. Although the sample-quality results are promising, they do not address this theoretical gap. Moreover, the experimental details are insufficient for independent reproduction, making it difficult to assess the empirical contribution on its own.

  2. The marginal-preservation claim in Proposition 3.4 does not hold in general. In Appendix B.5, the derivation treats the frozen random matrix SkSk⊤S_kS_k^\top as spatially constant in the unconditional Fokker–Planck equation. However, adaptedness alone does not justify this step. The relevant coefficient is E[SkSk⊤∣Xt=x]\mathbb E[S_kS_k^\top\mid X_t=x], which can depend on xx. To illustrate this issue, consider a stationary N(0,I2)\mathcal N(0,I_2) target with zero velocity, unit noise scale, and a frozen direction S=(cos⁡X0,1,sin⁡X0,1)S=(\cos X_{0,1},\sin X_{0,1}). Under Eq. (13), we obtain ddtE[Xt,2]∣0+=−12e−2≠0.\left.\frac{d}{dt}\mathbb E[X_{t,2}]\right|_{0^+}=-\frac12e^{-2}\neq0. This counterexample shows that an admissible direction can change the marginal distribution, contradicting the general claim in Proposition 3.4. While this invalidates the theorem as stated, it does not necessarily undermine the practical effectiveness of the proposed sampler.

Questions for authors
  1. How does Proposition 3.4 account for the counterexample above? If the proof conditions on the frozen direction, why is it valid to use the unconditional score in Eq. (13) for the resulting conditional distribution?

  2. Could the authors clarify the exact sequence of model evaluations for the 3-NFE and 5-NFE settings, including initialization, final updates, timestep schedules, and noise coefficients?

  3. Under the same solver settings and with matched noise energy, does the history-based direction perform better than fixed or independently sampled random rank-one directions?

Reviewer_E5SoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a training-free stochastic sampling framework for flow-matching models. It injects scalar Brownian noise into the sampling process and combines this with a projected score correction. The authors show the effectiveness of the proposed method in experiments on CIFAR-10, FFHQ, and ImageNet.

Critical strengths
  1. The method is computationally efficient, reusing existing solver stages to estimate the noise direction without additional model evaluations.
  2. The reported low-NFE improvements are substantial.
  3. The evaluation covers both pixel-space and latent-space generation and includes quality and diversity metrics.
Critical weaknesses
  1. Reproducibility concern. Table 12 mentions that “their conversion to the normalized coefficient is not documented in the archived experiment files.” This limits the reproducibility of the noise-scale ablation.

  2. Limited baseline coverage on SiT. The SiT comparisons include Heun2, Euler-Maruyama, and Heun. Comparisons with competitive multistep samplers under the same model and evaluation budget would better establish the relative performance of the proposed method in this setting.

  3. Insufficient comparison on diversity metrics. The authors should discuss the relationship to diversity-oriented flow methods such as DiverseFlow [1] and, where applicable, include empirical comparisons with this line of work.

  4. Incomplete validation of the error-based direction. Although Figure 3 compares several direction choices, it lacks a random unit-direction control with matched solver settings and noise strength. This comparison would help establish the additional benefit of the proposed error-based direction over a simpler random direction.

[1] DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows. Mashrur M. Morshed and Vishnu Boddeti. CVPR 2025.

Questions for authors

See Weakness section

Reviewer_tWTNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Given a vector field vtv_t, one can naturally associated to it a probability flow μt\mu_t by solving the continuity equation. A well known result is that μt\mu_t may also be regarded as the law of a stochastic differential equation with drift vt+stv_t+s_t, where sts_t is the score function of ptp_t. Though sampling from the SDE is desirable in many applications, e.g. for sampling diversity, it is computationally more costly. The present paper proposes SONGS, an algorithm that constructs a stochastic process driven by a one dimensional Brownian and motion and whose marginal flow is ptp_t. The purpose of this is to combine computational efficiency with the benefits of sampling from a SDE rather than from a ODE. The algorithm is tested across an array of classical benchmark, achieving very good performances at a moderate increase in computational cost with respect to deterministic samplers.

Critical strengths
  • The question of finding a low-rank stochastic differential equation for a given flow is natural and interesting.
  • Numerical experiments are extensive and suggest that SONGS performs very well in the low NFE regime.
Critical weaknesses
  • As I am not familiar with the two-stage small noise SRK method, the algorithm looks quite obscure to me. The authors should give some context and explain better why working with this version of the SRK method is a good choice

  • There is no theoretical convergence guarantee for the proposed method. One would like to see that working with one-dimensional noise and the particular choice of vev_{e} have beneficial effects on the convergence of the Euler scheme. As far as I understand, the findings of Appendix A.3.1. do not apply to SONGS directly, but I may be wrong here. Even if the theoretical analysis follows from known results assuming enough score regularity, I would appreciate having some result in this direction.

  • Overall, the theoretical novelty seems limited. Proposition 3.4 is essentially an adaptation of known results.

Questions for authors
  • What is the logic behind the choice of the noisy direction ve\mathrm{v}_e in SONGS? Is it true that is the direction along which the discrepancy between the true SDE and its Euler scheme in a short time-interval is the largest?
  • Is there a natural way to choose mm noisy directions if we work with mm-dimensional instead of one-dimensional noise?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.