acceptodds
Under review as a conference paper at ICLR 2027

Beyond Uniform Lipschitzness: Pathwise Jacobian Stability for Flow Matching Samplers

Abstract

Flow matching samplers integrate a learned velocity field. Existing error bounds often use its largest Jacobian norm over all states, even when generated paths visit only a small part of the space. We develop a framework that weights each velocity error and solver defect by the Jacobian growth that follows it between the compared paths. An exact error identity yields signed segment and tube bounds, with classical uniform Lipschitz estimates as a special case. In an explicit conditional flow matching example with nonzero velocity error, the signed bound equals the terminal Wasserstein error, while the classical bound is more than 500 times larger. We also determine what the population loss guarantees without a uniform Jacobian bound on the learned field. For a fixed population flow and a fixed linear growth class, the sharp worst case rate in state dimension is , where is the population velocity loss. Polynomial state weights reduce the logarithmic factor. Tests on trained samplers connect path growth to terminal sensitivity and use later growth to select paths for numerical refinement. The same flow derivative gives a weighted Poincar\'e inequality for the output law.

Primary area:
learning theory
Keywords:
Flow matching, Sampling theory, Stability analysis, Wasserstein distance, Numerical ODEs, Functional inequalities

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6izvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper argues that previous error bounds for flow matching samplers are too loose because they rely on a uniform Lipschitz norm in entire space. However, this raises the problem that this kind of estimation could also include regions that sampler never visit before.To answer this issue, the authors focus on Jacobian growth along the actual sampling trajectories and between the trajectories being compared. They showed that the final sampler's error depends not only on the velocity error, but also on how subsequent flow propagate. By considering for this error propagation and thus doing math, the author derives much tighter upper bounds on the "distance" between the generated and target distributions, which they validate through experiments.

Critical strengths

1: The paper is well-motivated: This paper addresses an important limitation of existing flow matching error bounds, and its motivation is quite easy to understand.

2: Strong theoretical contributions: This paper provides tighter error bounds without uniform Lipschitz assumptions and establishes sharp convergence rates from population loss.

3:Clear and convincing theoretical examples to support author's idea: The paper constructs examples showing that two models can have the same population loss and absolute Jacobian norms but result in very different final generation errors. This further support author's hypothesis.

Critical weaknesses

1: Authors only conduct the experiment only on the small-scaled and artificial experiments. However, I think these low-dimensional synthetic datas are not sufficient to demonstrate whether the proposed framework is effective for high-dimensional, real-world generative models. Even though it is theoretical paper, the author should at least try some small datasets, such as MINIST or CIFAR (if possible), in order to show this framework is valid more than toy examples.

2: The paper gives "Weighted population loss" (in the section 3). I think it is important invariance to further improve the upper bound for distributions (correct me if I am wrong). However, even though the author shows that weighted population loss improves theoretical guarantees, there is no result to test whether it improves actual generation performance.

Questions for authors

1: As I mentioned in the weakness part, could the authors evaluate whether the proposed framework remains effective on real-world image generation tasks, such as MNIST or CIFAR (CIFAR is even better), rather than only on low-dimensional synthetic datasets?

2: In Theorem 3, the upper and lower bounds involve the constants Cd,K,LC_{d,K,L} and cd,K,Lc_{d,K,L}, respectively. Could the authors provide any hint for numerical ranges for these constants in different dimensions? For example, when applying flow matching to high-dimensional datasets such as ImageNet (256×256256 \times 256), could Cd,K,LC_{d,K,L} become extremely large as the dimension increases, making the upper bound too loose to be practically informative?

3: The standard flow matching objective minimizes the velocity prediction error. Although the proposed weighted population loss is mainly used for theoretical analysis in term of upper bound, it seems possible to incorporate a similar weighting scheme into the training objective:

Lweighted=Et,x0,x1[(1+∣xt∣)a(v(t,xt)−(x1−x0))2]L_{weighted} = E_{t, x_0, x_1} [(1+|x_t|)^a (v(t,x_t) - (x_1-x_0))^2]

I wonder whether this weighted training objective could improve the final generation quality compared with standard flow matching. Have the authors considered evaluating this experimentally?

If the author could solve these questions, I would consider raising my score.

1 comment
AuthorsOpenReview ↗

Real-world datasets, constants, and weighted training

Thank you for your careful review and your constructive questions about real-world image generation, dimension-dependent constants, and whether weighted training improves generation quality. We respond to each point below.

W1 / Q1. Evaluation on real-world image generation tasks

We have added experiments on MNIST and CIFAR-10, the two datasets you suggested, in Sections 5.2–5.3. CIFAR-10 compares a trained reference model with three fixed weight modifications. Across 300 paired comparisons, signed estimates are 8.22–12.76 times endpoint RMSE (Table 2).

On MNIST, 144 measurements from 18 checkpoints compare path growth with estimated terminal Heun-map operator norms. The median log gaps correspond to factors of about 2.9 for signed growth and 586 for the operator-norm integral (Figure 5). Appendix I.1 gives the settings and precision checks.

Q2. Dimension dependence of the upper and lower constants

Yes, the sufficient upper constant can be extremely large. Appendix H.2 now gives its formula. For K=L=1K=L=1, log⁡10C\log_{10} C is 8.61 at d=1d=1, 1,843.43 at d=784d=784, 7,635.26 at d=3072d=3072, and 576,048.31 at d=3×256×256d=3\times256\times256. These values illustrate the scale of the proof constant.

The sharp rate describes how error depends on loss when dimension and growth constants are fixed. For an individual model, Theorem 2 retains its residual and the growth along the compared paths. For the lower bound, the proof establishes a positive constant independent of loss for fixed K≥K∗K\ge K_* and sufficiently small loss budgets. This establishes the matching dependence on loss. The numerical values above describe the explicit upper constant.

W2 / Q3. Does weighted training improve generation quality?

We trained standard CFM, state weighted CFM with a=2a=2, and a time weight control on MNIST, using three paired seeds and 20,000 updates. State weighting lowers feature Fréchet distance in two pairs and raises it in one. The mean change is −2.52% relative to standard CFM, with a seed standard deviation of 6.61 percentage points. Relative to the time control, it is +0.75% ±1.17 percentage points. Weighted validation risk decreases in one of three pairs (Section 5.4 and Table 6).

With a known 32-dimensional reference velocity, weighted residual, endpoint RMSE, and tail RMSE decrease in all three paired runs, by 3.11%, 1.73%, and 3.83% on average (Appendix I.2). The theorem links an attained weighted loss to an error guarantee. The training experiments report the loss and sample quality actually reached.

Reviewer_YzgsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how a velocity prediction error grows or shrinks during the remaining steps of flow matching sampling. It bounds the final generation error using derivative information between the ideal and learned trajectories, instead of using the worst case over the entire space. It also proves matching upper and lower bounds on generation error given a training-loss budget, under specific growth assumptions. Validation includes explicit low-dimensional examples, small trained models, and experiments that select samples for additional numerical refinement.

Critical strengths
  1. The path-dependent bounds avoid irrelevant worst-case regions and can remain informative even when global bounds are excessively large or infinite.

  2. The matching upper and lower bounds establish the best possible worst-case relationship between population training loss and generation error under the stated assumptions.

Critical weaknesses
  1. Limited validation in higher dimensions. The main sensitivity and refinement experiments use low-dimensional models. The 64-dimensional experiment focuses on Jacobian norm estimation, without testing whether the diagnostics predict generation error or guide refinement. Their effectiveness in realistic high-dimensional generative models therefore remains insufficiently demonstrated.

  2. Computational benefits are not established. The refinement comparison matches retained integration steps rather than total computational cost. Although the method retains 19.2 steps on average, the described implementation requires 23.4 velocity evaluations with reuse, plus 16 Jacobian norm evaluations per path. Comparisons at equal total runtime are needed to determine whether the accuracy gains justify this overhead.

  3. The connection to practical model evaluation remains unclear. The sharper bounds depend on unknown population trajectories and derivative information between the population and learned paths. These quantities are generally unavailable in practice, and the measured path-growth diagnostics primarily capture sensitivity to initialization. The paper does not establish whether these accessible diagnostics can reliably assess generation quality or distinguish better-trained models beyond what validation loss already reveals.

Questions for authors

Please see weaknesses

1 comment
AuthorsOpenReview ↗

High-dimensional diagnostics, cost, and model evaluation

Thank you for your careful review and constructive questions about diagnostics in high dimensions, the total cost of refinement, and practical model evaluation. We respond to each point below.

W1. Error diagnostics and refinement in higher dimensions

We have added CIFAR-10 experiments in 3,072 dimensions and MNIST experiments in 784 dimensions (Sections 5.2–5.3). All 300 CIFAR numerical comparisons follow the predicted ordering. Signed estimates are 8.22–12.76 times paired endpoint RMSE, while tube estimates are 387–724 times larger than signed estimates (Table 2).

The direction decomposition explains most of the remaining signed gap: maximizing growth over directions contributes factors of 6.25–9.31, and replacing the residual projection by its norm contributes 1.32–1.44. The underlying identity reconstructs endpoint error within 0.006% (Appendices C.2 and I.1). Figure 5 also compares path growth with terminal operator norm estimates on MNIST. The high-dimensional results address error bounds and sensitivity. Section 5.4 examines refinement on five trained two-dimensional models, with the full cost comparison discussed below.

W2. Refinement accuracy and total computational cost

We now report full runtime, including the pilot solve, Jacobian calculations, scoring, selection, and reruns. For 1,500 paths, growth selection takes 0.05927 seconds with RMSE 0.06610. Heun with 16 steps takes 0.02273 seconds with RMSE 0.01099, and RK4 with eight steps takes 0.02279 seconds with RMSE 0.00349 (Table 8).

The tested Heun and RK4 settings are both faster and more accurate. Section 5.4 therefore focuses on identifying paths with large Euler error: the 20% selected by growth weighting contain 82.3% of squared error, versus 28.5% for local defects alone. Appendix I.4 distinguishes the 19.2 output steps from the full cost of 23.4 velocity evaluations plus 16 Jacobian norm evaluations.

W3. Unknown population quantities and practical model evaluation

The fixed CIFAR reference lets us evaluate both paths and their residual. This measures error relative to the reference model's output law. Error relative to the data law also includes that model's own error. For a single flow, the path quantities measure initialization sensitivity, which the MNIST experiment checks.

To examine whether the diagnostics help assess generation error, we also tested error ranking. The residual integral has a higher correlation with endpoint error than the signed estimate in each CIFAR comparison (Appendix I.1). For practical model evaluation, these results support using the diagnostics to analyze error propagation between known flows and sensitivity within a flow. In the CIFAR comparisons, the residual integral gives the stronger error ranking.

Reviewer_gK9nOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The manuscript studies the effect of approximation errors on flow matching models. It establishes bounds that are based on relevant generation paths, unlike prior works which assume worst case sensitivity everywhere. Additionally, the theory derives lower and upper bounds for the flow matching model generation accuracy. The theoretical results are supported by synthetic experiments.

Critical strengths
  • Novel theoretical results about the approximation errors on flow matching models
  • Empirical support of the theory on synthetic data
Critical weaknesses
  • It is unclear if the theoretical claims agree with broader empirical observations on flow matching models trained on non-synthetic datasets
  • The readability of the manuscript is poor, the terminology and the wording used are hard to follow, hence for the reader it might be hard to understand the contributions, significance, impact and practical implications of the work
Questions for authors

Please see weaknesses.

1 comment
AuthorsOpenReview ↗

Non-synthetic datasets and readability

Thank you for your careful review and for asking whether the theory agrees with observations on non-synthetic datasets. We have added experiments on models trained on CIFAR-10 and MNIST and revised the presentation. We respond to both concerns below.

W1. Agreement with observations on non-synthetic datasets

Sections 5.2–5.3 now test the framework on models trained on two non-synthetic datasets, CIFAR-10 and MNIST. On CIFAR-10, we compare a trained reference model with three fixed weight modifications, using the same 100 initial states for each. The signed estimates are 8.22–12.76 times the paired endpoint RMSE. Tube estimates are 387–724 times larger than signed estimates (Table 2). The error here is relative to the reference model.

On MNIST, we compare path growth with the estimated terminal Heun-map operator norm in 144 measurements from 18 checkpoints. The median log gaps correspond to factors of about 2.9 for signed growth and 586 for the operator-norm integral (Figure 5). Appendix I.1 gives the numerical checks. These results support the predicted error propagation and sensitivity relationships in models trained on non-synthetic data.

W2. Readability, contributions, and practical implications

The main contribution is an error bound that retains each velocity residual and the later Jacobian growth between the compared paths. The introduction now separates two questions: what this information tells us about a given sampler, and what population loss alone guarantees under fixed growth conditions.

Section 2 explains the CFM loss identity before the generation bound and distinguishes sensitivity along one path from error between two paths. Figures 1–2 illustrate this distinction. We have also simplified headings and unified notation. Sections 5.2–5.4 explain the uses tested by the experiments: comparing error bounds, measuring initialization sensitivity, and locating paths with large numerical error.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.