- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Target-Guided Flow Matching (TGFlow) that modifies the source distribution from an isotropic Gaussian to honor the second-order statistics (e.g., covariance structure) of target distribution. It theoretically shows editing the source can affect the learning of the velocity field, and empirical evidences on CIFAR-10 and ImageNet-{32,64} show lower Frechet Inception Distance (FID) score and Neural Function Evaluation (NFE) than typical diffusion / flow matching methods.
Critical strengths
- Since flow matching gives the freedom to choose an arbitrary source distribution, the line of thinking to design a source that's better align with the target is intuitive. The authors, in their writing, unveil the intuition very nicely.
- The authors also identify the limitations of prior source-design works and propose a solution to them.
Critical weaknesses
- Both theorem 1 and 2 are restating known flow-matching fundamentals from Lipman et al. (2024) and Lai et al. (2025).
- Theorem 3 is trivial as it follows with standard Gaussian KL. Also, it bounds for a untrained mental-model network, not really the KL between a learned/predicted target distribution and the actual target distribution.
- Section 3.3 is not really discussing any insights of extending TGFlow to any arbitrary source distributions as the section name suggests; instead, it reads more like an introduction of data coupling via optimal transport.
- The extension to non-Gaussian source distribution is an overclaim. There is no conceptual illustration of how TGFlow works beyond non-Gaussian source distributions (e.g., heavy-tail, exponential), and there is only one toy experiment where the source distribution is "moons" (with limited souruce estimation details), which obviously falls short to support the claim.
- There is no other baseline methods of non-Gaussian source distributions in the experiments, not even any of the source-design methods the paper cites; besides, another reasonable baseline should be ) where is a rescaling factor to matched the TGFlow's estimated variance level (i.e., match in total energy) so that the readers can know the benefits come from the scale or the shape.
References
- Lipman et al. (2024) Flow Matching Guide and Code.
- Lai et al. (2025) The Principles of Diffusion Models
Questions for authors
Please refer to the critical weaknesses that I raised. I think the most important baseline shall be the matched variance-level rescaled isotropic Gaussian, while benefits over other source-design methods are secondary. The remaining concerns are conceptual. There is one more conceptual thing that I want to point out which is the practicality of obtaining second-order statistics in a reliable manner for high-dimensional data, whose difficulty might be not easier than generative modeling itself.
11 comments
Assumption A2 (Relative Deviation Bound for Conditional Covariances). There exists a common constant such that, for all and
\left\\| (G_{t}^{\Sigma})^{- 1/2}C_{t}^{\Sigma}(x)(G_{t}^{\Sigma})^{- 1/2} - I \right\\|_{op} \leq \zeta.\quad\quad\text{(R)}
To state the theorem, define the reference Lipschitz constant and its time integral by
and
Theorem 1 (Comparison of Time-Integrated Lipschitz Constants Across Source Distributions). Under the above setting and Assumptions A1–A2, the ideal marginal velocity fields satisfy
If the target's marginal variances are not all equal, then
If additionally
then
\int_{0}^{1}{L_{t}^{\Lambda}\\, dt < \min\left\\{ \int_{0}^{1}{L_{t}^{I}\\, dt,\\,\int_{0}^{1}{L_{t}^{cI}\\, dt}} \right\\}.}
Proof. Let
Since and
For differentiating the conditional mean under the Gaussian observation model gives
Consequently,
Define
Because and are diagonal,
Thus,
Condition (R) implies
Multiplying on both sides by we obtain
\left\\| \Sigma^{- 1/2}\left( C_{t}^{\Sigma}(x) - G_{t}^{\Sigma} \right)\Sigma^{- 1/2} \right\\|_{op} \leq \zeta\max_{i}\frac{(G_{t}^{\Sigma})_{ii}}{\Sigma_{ii}}.
It follows that
\left\\| \left( C_{t}^{\Sigma}(x) - G_{t}^{\Sigma} \right)\Sigma^{- 1} \right\\|_{op} \leq \zeta\sqrt{\kappa_{\Sigma}}\max_{i}\frac{(G_{t}^{\Sigma})_{ii}}{\Sigma_{ii}}.
Moreover,
Therefore,
\left\\| \nabla_{x}v_{t}^{\Sigma}(x) - A_{t}^{\Sigma} \right\\|_{op} \leq \zeta\sqrt{\kappa_{\Sigma}}\frac{tr_{\Sigma}}{\sigma_{t}(\sigma_{t}^{2} + t^{2}r_{\Sigma})}.
For a continuously differentiable velocity field, the smallest global spatial Lipschitz constant equals the supremum of the operator norm of its Jacobian. Hence,
At independence gives
Using with yields
\int_{0}^{1}{\frac{tr_{\Sigma}}{\sigma_{t}(\sigma_{t}^{2} + t^{2}r_{\Sigma})}\\, dt} = \int_{0}^{1/\sigma_{\min}}{\frac{ur_{\Sigma}}{1 + u^{2}r_{\Sigma}}\\, du} \\\\ = \frac{1}{2}\log\left( 1 + \frac{r_{\Sigma}}{\sigma_{\min}^{2}} \right). \end{array}$$ Continued in the next reply.Hence,
To establish positivity of the reference gaps, define
For
The reference quantities satisfy
Since for at least one coordinate,
If the marginal variances are not all equal, let Since we have
for all sufficiently small Thus, on an interval of positive length, implying
Also, writing we have and
By strict monotonicity of for every
Thus, at least one of the two endpoint values has an absolute value strictly greater than Consequently,
and hence
Finally, for each condition (C) gives
\int_{0}^{1}{L_{t}^{\Lambda}\\, dt} \leq J_{\Lambda} + \zeta H_{\Lambda} \\\\ < J_{B} - \zeta H_{B} \\\\ \leq \int_{0}^{1}{L_{t}^{B}\\, dt.} \end{array}$$ Therefore, $$\int_{0}^{1}{L_{t}^{\Lambda}\\, dt < \min\left\\{ \int_{0}^{1}{L_{t}^{I}\\, dt,\\,\int_{0}^{1}{L_{t}^{cI}\\, dt}} \right\\}.}$$ This completes the proof. $\square$ Theorem 1 concerns ideal velocity fields, whereas the stability bound in \[1\] depends on a spatial Lipschitz upper bound for the learned velocity field. Condition (R) controls deviations of the target’s conditional second moments from the Gaussian reference, while condition (C) ensures that these deviations do not eliminate the comparison advantage of the TGFlow source in the reference setting. To connect this ideal-field analysis to learned models, we impose an additional regularity condition beyond Assumptions 1–3 in \[3\]: across different source distributions, the Lipschitz constant of the learned velocity field does not exceed that of the corresponding ideal velocity field plus a common nonnegative, integrable remainder. Under this condition and a common upper bound on the population velocity-field approximation error, the stability argument in \[3\], together with the terminal residual term, yields a Wasserstein-2 generation-error bound and allows us to compare the resulting upper bounds across source distributions.
Weakness 1: Both theorem 1 and 2 are restating known flow-matching fundamentals from Lipman et al. (2024) and Lai et al. (2025).
We thank the reviewer for the detailed comments. We agree that the original Theorems 1 and 2 build upon established theoretical foundations, including ODE flows, the continuity equation, and affine conditional flow matching. Their primary contribution lies in providing theoretical insights into our method, rather than introducing entirely new theoretical results. Specifically, their contributions are twofold: the original Theorem 1 clarifies the dependence of probability paths on the source distribution, whereas the original Theorem 2 establishes that TGFlow’s source distribution adjustment preserves the analytical forms of the existing conditional velocity field and training objective. To accurately reflect the relationship between these results and prior work, we will no longer present them as standalone theorems in the revised manuscript. Instead, we will present the original Theorem 1 as a Remark based on reference [1] and the original Theorem 2 as a Corollary based on existing reference [2].
[1] Lipman, Yaron, et al. "Flow Matching for Generative Modeling." ICLR 2023 (2023).
[2] Lai, Chieh-Hsin, et al. "The Principles of Diffusion Models." CoRR abs/2510.21890 (2025).
Weakness 2: Theorem 3 is trivial as it follows σmin2Λ≤σmin2I with standard Gaussian KL. Also, it bounds for a untrained mental-model network, not really the KL between a learned/predicted target distribution and the actual target distribution.
We thank the reviewer for pointing out this issue. We agree that, under the Gaussian target distribution assumption, Theorem 3 follows from the standard Gaussian KL divergence formula and the covariance ordering relation σmin2Λ≤σmin2I. Nevertheless, this result provides an analytical explanation of the benefit of constructing the source distribution using the target’s coordinate-wise variances: under these assumptions, the KL divergence between the terminal marginal distribution of TGFlow’s ideal probability path and the target distribution is no greater than the corresponding KL divergence obtained with the standard isotropic source. We also acknowledge that this result does not account for velocity-field learning error and therefore does not fully characterize the discrepancy between the actual generated distribution and the target distribution. To address this limitation, following the suggestion of Reviewer 1, we have added two theoretical results based on the stability analysis framework in reference [3]. These results show that, under the assumptions explicitly stated in the two new theorems, TGFlow’s Wasserstein-2 generation-error upper bound is no greater than the corresponding bounds obtained with either the standard isotropic source or the isotropic source with the same total variance as TGFlow’s source, and is strictly smaller under the specified conditions.
Assumption A1 (Moment conditions on the target distribution). The target distribution q is a probability distribution on Rd with finite second moments and strictly positive marginal variances in all coordinates, that is,
EY∼q∣Y∣2<∞,di:=VarY∼q(Yi)>0,i=1,…,d.
Define
D=diag(d1,…,dd),M=maxidi,Λ=MD,c=dtr(Λ).
Here, D collects the marginal variances and need not equal the full covariance matrix of q.
For each Σ∈Λ,I,cI, independently sample
X0Σ∼N(0,Σ),Y∼q,
and define the interpolation path
XtΣ=σtX0Σ+tY,t∈[0,1],
where
σt=1−at,a=1−σmin,0<σmin<1.
Let
ptΣ=Law(XtΣ),
and define the ideal marginal velocity field and its smallest global spatial Lipschitz constant by
vtΣ(x)=E[Y−aX0Σ∣XtΣ=x],LtΣ=Lipx(vtΣ).
Next, introduce the conditional covariance matrix and its Gaussian reference matrix:
CtΣ(x)=Cov(Y∣XtΣ=x),GtΣ=(D−1+σt2t2Σ−1)−1.
The above conditional moments are understood as the canonical versions induced by the Gaussian observation kernel.