acceptodds
Under review as a conference paper at ICLR 2027

EUCLIDEAN DISTANCE-BASED FLOW MATCHING

Abstract

Flow Matching has recently become popular as a technique for generative modeling with applications in many domains. In the standard Flow Matching formulation, we interpret the vector field as the solution of a least-squares regression problem, which is known to be sensitive to contamination. Therefore we leverage the viewpoint of robust statistics and replace the squared Euclidean distance with the Euclidean distance as a regression objective. This seemingly minor change strongly affects the properties of Flow Matching. We provide a probabilistic interpretation of the new regression target as a conditional geometric median and use this characterization to study distributional robustness of the Euclidean Distance-based Flow Matching. We analyze the mathematical properties of this changed objective and the effect of a contaminated target distribution on the involved conditional velocity distributions. We discuss the challenge of directly characterizing the distribution that our method induces. In specific contamination settings we show theoretically that our method yields an improvement for severe contamination. Finally, we present numerical experiments conducted on the CIFAR-10 image dataset where a proportion of the samples has been contaminated. Our Euclidean distance-based Flow Matching shows a clear advantage in scenarios of severe contamination of the target distribution.

Primary area:
generative models
Keywords:
Flow Matching, Robust Statistics, Geometric Median

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Umg7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a modification of the flow matching loss where the squared Euclidean norm is replaced with the Euclidean norm. The minimizer of the resulting loss is a conditional geometric median of the distribution of X1X_1 given XtX_t, which is known to be more robust to corruptions of the data distribution than the conditional mean. The authors derive sufficient conditions under which the minimizer of the Euclidean distance flow matching loss coincides with the minimizer of the traditional flow matching loss, which ensures that Euclidean distance flow matching generates the correct distribution. The authors also prove robustness bounds for Euclidean distance flow matching when the data distribution is corrupted, both for the velocity field and the generated distribution. Experimental results on image datasets, comprising different mechanisms for corrupting the data, complement these robustness guarantees.

Critical strengths

The paper studies an important problem, namely improving the performance of generative models under imperfect training data. The method proposed by the paper is mathematically principled and yields performance guarantees over standard flow matching in certain settings.

Critical weaknesses

The main limitation is that the experimental results do not seem very convincing. Other than for the random noise experiment, the results obtained by Euclidean distance flow matching seem comparable or even worse than those obtained by standard flow matching. Such numerical results make it difficult to endorse a methodology that requires training a velocity field from scratch (as opposed to modifying a pre-trained model).

Another (less significant) weakness is that it seems difficult in general to characterize whether the generated distribution (with the exact Euclidean distance velocity field, under exact integration) is equal to the data distribution. This is guaranteed when the mean and the geometric median of the conditional distribution given XtX_t are equal for each time t∈[0,1];t \in [0,1]; the authors prove this holds for Gaussian data, but it is unclear beyond Gaussian data when this holds.

Questions for authors

To be sure that I understand: in your experiments, the architecture and training data are the same for training in both the standard flow matching and Euclidean flow matching settings?

Minor comments:

  • I found the proof of Theorem 4.2 quite difficult to follow, even though it was well-written on a per-line basis. It would be helpful to break the proof up into several key steps and explain how the steps combine to prove the claim.
  • The upper bound in Theorem 4.2 can be simplified in terms of the magnitude of the contamination:

W1(Pcont,δ0)=W1((1−ϵ)δ0+ϵδb,δ0)=ϵ∥b∥.W_1(P^{\textrm{cont}}, \delta_0) = W_1((1-\epsilon)\delta_0 + \epsilon \delta_b, \delta_0) = \epsilon \|b\|.

Writing the result in this way might make the bound clearer.

  • Similarly, the bound in Theorem 4.3 can be further upper bounded by

W1(PRcont,P)=W1((1−ϵ)P+ϵδbR,P)≤ϵEX∼P[∥X−bR∥].W_1(P_R^{\textrm{cont}},P) = W_1((1-\epsilon)P + \epsilon \delta_{b_R}, P) \leq \epsilon \mathbb{E}_{X \sim P}[\|X-b_R\|].

2 comments

Weaknesses:

Regarding experimental results:

On an established flow-matching architecture (on this data set) we consider these results convincing since we achieve a lower FID than standard FM for five of the seven contamination mechanisms across all tested contamination rates. The gains are moderate and grow with ϵ\epsilon, while the standard objective is better for both blur mechanisms. We discuss this in our answer to question 1 from reviewer 3mqt.

Regarding training from scratch: The L2,1L_{2,1} objective changes only the loss, at the same training cost as standard FM, and it targets the setting in which the available training data are themselves contaminated. Its population minimizer does not depend on the initialization, so the loss can also be used to fine-tune a pre-trained model on contaminated data although we have not evaluated this.

Regarding the characterization: A general characterization would be valuable, but it appears to be out of reach. Apart from the Gaussian case, we cannot characterize when the two fields coincide. Note that this is only sufficient, not necessary, for reproducing the target. In general, however, we do not expect the L2,1L_{2,1} flow to reproduce non-Gaussian targets exactly. As in Theorem 4.2, it can shrink minority components of the target, whether they are contamination or not. This is the trade-off stated in the introduction: We give up exact reproduction of the observed distribution in exchange for robustness to contamination.

Question: Yes, the neural network architectures and hyperparameter choices are fixed. Also, both methods in one run see the same training data, because the data pipeline and seeds are controlled.

Minor comments: We thank the reviewer for these helpful suggestions and will incorporate them. In particular, we will restructure the proof and provide explanations of how the statements in the proof will be used.

Thank you for your reply. Your response helps me understand the trade-off that your model makes, and when it might or might not make sense to use it. I find the contribution sound and interesting, but both the theory and the experiments have weaknesses (as described in my review). I will maintain my score, as I think the paper is closer to borderline.

Reviewer_U3SYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a simple but effective change to the original Flow Maching (FM) training objective by replacing the original squared Euclidean distance with the Euclidean distance, in order to improve the robustness of generative models when faced with corrupted training data. It is shown that the population-optimal regression target for the new objective corresponds to the geometric median of the conditional velocity distributions. The relationship between this field and the standard velocity field is analyzed, and it is also shown that under two specific types of contamination models, the Wasserstein distance between the generated distribution and the clean target distribution is improved. Experiments conducted on the CIFAR-10 dataset compare the generative FID scores obtained using two different loss functions under both clean training conditions.

Critical strengths

S1: The paper possesses a clear statistical interpretation.

By replacing the squared loss in flow matching with an Euclidean distance loss, the optimal velocity field shifts from the conditional mean to the conditional geometric median, which is a change that provides a direct basis for mitigating the influence of contaminated samples on velocity predictions.

S2: The paper presents non-trival results at the distribution level.

For two-point targets and single-point contamination located far away, the paper demonstrates that modifying the loss function brings the generated distribution closer to the clean target. It indicates that under certain conditions, local robustness can translate into an improvement in the final generation quality.

Critical weaknesses

W1: There remains a significant gap between theoretical guarantees and the robustness claimed in practice.

Lemma 3.1 characterizes the population-optimal regression target while Theorem 3.4 and Lemma 4.1 primarily bound the velocity field. The paper does not provide a general characterization of the probability paths induced by the modified loss, nor does it offer error bounds on the terminal distribution that apply to general clean and contaminated distributions. Theorem 4.2 provides explicit results for a two-point objective. Theorem 4.3 requires the clean distribution to be bounded and the single-point contamination to be sufficiently distant, while also assuming the existence of a solution to the corresponding ODE. These are valid results for special cases, but they do not explain the overall pattern observed on the CIFAR10 experiments and more possible extended results.

W2: The experimental scope is insufficient to assess robustness across data distributions.

All experimental results are based solely on the CIFAR-10 dataset, lacking validation on more complex datasets or diverse generative tasks. Furthermore, the forms of contamination employed are limited to predefined noise or image transformations. The paper does not account for complex, mixed-source contamination, semantic outliers, or distribution shifts typical of real-world collected data. In such scenarios, observations that deviate from common samples could potentially represent valid, rare patterns, and it remains unclear whether the proposed method might erroneously suppress them.

Questions for authors

Q1: Does the improvement extend beyond CIFAR-10 and pixel-level corruptions?

Please evaluate both losses on at least one more complex image dataset, (e.g. ImageNet-64), under clean training and contaminated training. For the latter, please specify the clean distribution (P) and construct a semantic contamination distribution (Q)—for example, images from held-out classes or a distinct image source—then train on ((1−ϵ)P+ϵQ)((1-\epsilon)P+\epsilon Q) at stated contamination rates. Please report FID against clean (P), together with a measure of how often generated samples resemble (Q) and whether coverage of (P) declines. These results are central to assessing whether the observed robustness extends beyond the particular CIFAR-10 corruptions studied here.

Q2: What can be established about the generated distribution beyond the special cases in Theorems 4.2 and 4.3?

For a nontrivial clean distribution and contamination distribution, can you either derive a bound on the distance between the induced distribution and (P), or give a concrete counterexample where the modified objective is worse than standard FM?

Q3: How does the Euclidean-distance loss compare with other robust losses under the same FM architecture and training budget? e.g, pseudo-Huber Loss

2 comments

W1:

We agree that our guarantees only cover special cases and we cannot give characterizations in general. The velocity bounds provided in Theorem 3.4 and Lemma 4.1 might carry over to a bound on the generated distribution if the field were Lipschitz, which does not generally hold and even in that case an improvement would not be clear. Thus we work on two separate levels instead. In Section 4 we are studying the velocity fields to analyze how contamination enters the geometric median. For controlling generated distributions we restrict our analysis to special settings where it is feasible. The analysis of the velocity fields suggests that the posterior contamination probability $\gamma_t(x) $ is an important quantity and we use it to prove the distribution results. The irregularity of the geometric-median field, which can jump where the posterior majority switches, is why we need the assumptions in Theorem 4.3. Regarding CIFAR-10, we agree that our theory does not explain the experimental pattern directly, since it does not cover the contaminations we use, although it offers some insight into when our method might perform better. In our response to Reviewer 3mqt (Q1), we give our intuition for the difference between noise and blur.

W2: We agree that the experimental scope is limited. We address ImageNet in our response to Q1. While it is true that our contamination settings are somewhat limited, they are designed as controlled experiments. Each of the mechanisms we chose is applied in a separate experiment, which allows us to gain insight into how the methods respond to different types of contamination. Several of the mechanisms are taken from Li and Dong (2026), who combine them into a single mixture. Evaluating them separately reveals the dependence of the outcome on the contamination, including the unfavorable blur experiments. Also, four of them are among the standard corruption types of the CIFAR-10-C dataset (Hendrycks and Dietterich,[ https://openreview.net/forum?id=HJz6tiCqYm ]): Gaussian noise, Salt-and-Pepper, Motion Blur and Gaussian Blur and they model degradations that occur when real images are collected, such as noise in low light, bit errors and camera motion. We agree that it would be useful to study additional contamination types. Regarding rare valid patterns, we agree that they might also be suppressed by our method, as demonstrated in the setting of Theorem 4.2. This is the trade-off stated in the introduction. On clean CIFAR-10 we observe no loss in quality, but the FID does not directly measure the coverage of rare modes.

Q1:

We answer this question in the general section.

Q2:

At the moment we are not able to provide further concrete examples, but we agree that they would be interesting to have and seek to provide those in the future. Under the stated assumptions our Theorem 4.3 shows such an improvement for nontrivial clean distributions with remote Dirac contamination, and as described in the answer to Q3 from reviewer 3mqt we hope that we can extend it to more general contaminating distributions.

Q3:

We answer this question in the general section.

Thanks very much for these additional experiments and clarifications. I believe the extra ImageNet-100 results provide some useful evidence beyond CIFAR-10. The Huber and pseudo-Huber comparisons address my request for robust-loss baselines.

However, my main concerns remain. The ImageNet experiment uses a 100 class subset at 32×3232\times32 resolution, and the reported semantic-contamination result is a modest FID improvement. Without measuring how often generated samples resemble the contaminating distribution (Q), or whether coverage of the clean distribution (P) declines, it is difficult to tell whether the method selectively suppresses contamination or also suppresses valid rare modes. As the authors themselves note, FID alone does not resolve this question.

On the theoretical side, the rebuttal confirms that the distributional results remain restricted to the special cases in Theorems 4.2 and 4.3. The explanation about possible discontinuities of the geometric-median field is helpful, but it does not establish what happens to the generated distribution under the more general contamination settings used in the experiments.

I highly appreciate the authors' effort in the rebuttal stage. They address several of my concerns, although my main questions about the generated distribution and coverage remain. I believe this is a borderline paper and encourage the authors to pursue these questions further. I maintain my score of 2.

Reviewer_3mqtOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper mainly focuses on replacing the squared Euclidean distance with the Euclidean distance as a regression objective in flow matching to tackle the issue of target contamination. The paper characterize the population-optimal Euclidean distance flow matching vector field as a conditional geometric median. The paper analyzes the proposed method achieves an improvement in Wasserstein distance for specific contamination cases. Finally, the paper experiments on contaminated CIFAR-10 datasets to show the effectiveness of the proposed method.

Critical strengths
  1. The paper is clearly organized. The paper first compares the L2,1L_{2,1} velocity field with the standard L2,1L_{2,1} velocity field, and then proceed to the contaminated scenario and discuss from the velocity field discrepancy to the distributional discrepancy.

  2. The topic of the paper is intriguing. Using the geometric median to tackle the robustness issue in flow matching is a simple yet not fully explored approach.

Critical weaknesses
  1. The main drawback of this paper is that it does not provide a thorough comprehensive comparison between the standard method and the proposed method both theoretically and empirically, reducing the informativeness and the credibility of the proposed method. In Section 4.1, the paper does not give further discussions on what kind of P,QP, Q and ϵ\epsilon will either vt1,ϵv_t^{1,\epsilon} or vt2,ϵv_t^{2,\epsilon} be closer to vt2v_t^2, as the authors admit that “those bounds do not directly compare to each other”. The heuristic “including less factors that depends on the contamination distribution” advantage may not be credible since one can argue that the discrepancy between vt2,ϵv_t^{2,\epsilon} and the clean velocity field does not blow up with the factor γt(x)\gamma_t(x) while vt1,ϵv_t^{1,\epsilon} can. In Section 4.2, the examples are restricted to delta contamination distributions, which does not cover the cases in experiments. In experiments, Table 3 shows that the standard method performs better for both Gaussian blur and Motion blur, the possible explanation provided is that “the blurring we applied produces a contamination that is less separated from the original distribution”. “Less separated” is not an informative claim, and it would be beneficial to discuss why are the results different between “Gaussian noise” and “Gaussian blur”. More experiments on other datasets like ImageNet will also enhance the effectiveness of the proposed method.

  2. The presentation of the paper can be further improved. It would be more informative to include the actual difference between W1(ρ1,δ0)W_1(\rho_1,\delta_0) and W1(Pcont,δ0)W_1(P^{\text{cont}},\delta_0) rather than a simple "<<" notation in Theorem 4.2 and Theorem 4.3.

Questions for authors
  1. Could the authors explain why the proposed method a the standard method perform differently on “Gaussian noise” and “Gaussian blur” contamination? It would be beneficial to further explain the relationship between the delta contamination distribution case covered in theoretical reasoning and the real contamination distributions in CIFAR-10 experiments.

  2. What is the performance of the proposed method on other datasets like ImageNet?

  3. Will Theorem 4.3 also hold for bounded contamination distributions beyond the point-mass contamination distribution? How will the results of Theorem 4.2 and Theorem 4.3 change if we change the Wasserstein-1 distance to other more commonly used metrics like the Wasserstein-2 distance?

2 comments

W1:

While there is a discrepancy between theory and practice and 2(1−γt(x))/(1−2γt(x))2(1-\gamma_t(x))/(1-2\gamma_t(x)) blows up as γt(x)→1/2\gamma_t(x) \to 1/2, our Lemma 4.1 is supposed to analyze the effect of the contamination in the regions where it is a minority in the posterior distribution. On those, the L2,1L_{2,1} bound depends on the contamination only through γt(x)\gamma_t(x) whereas the deviation in the squared fields also scales with the distance between the contaminating and clean conditional means. However, the region where γ<1/2\gamma < 1/2 also depends on the contamination, so we cannot expect the L2,1L_{2,1} field to be closer everywhere. In particular, close to the contamination, where γ>1/2\gamma > 1/2, the geometric median is dominated by the contamination. A general characterization of which method is favored seems out of reach. Since we cannot characterize the generated distribution in general, in Section 4.2 we focus on certain special settings where this is possible. There, the right-hand sides of Theorems 4.2 and 4.3 are exactly the error of standard FM, so the theorems compare the two methods directly.

Experiments: We agree that "less separated" is not informative. In our answer to Question 1, we explain why the results differ between Gaussian noise and Gaussian blur and how the Dirac setting relates to the experiments. ImageNet is addressed in our general response.

W2: Some statements are presented as weaker and less informative than what we have proven and we will adjust them accordingly. Also, we will try to improve the overall structure and make our claims more precise.

Q1:

In the theory we identify a property that our method alleviates contamination if it lies away from the clean data. For the Gaussian noise and Gaussian blur experiments, our intuition is that the contaminations differ in where they place images relative to the clean data. Gaussian noise adds independent pixel-wise perturbations, unlike the variation between clean CIFAR-10 images, so noisy images move away from the clean data. Blurred images on the other hand still resemble natural out-of-focus images and we expect them to be closer to the clean data, which is where the median might be worse than the mean.

If the question is why the methods do not coincide, "Gaussian" refers to the corruption, not the target. The contaminated target (1−ϵ)P+ϵQ(1-\epsilon)P + \epsilon Q is not Gaussian and thus Corollary 3.3 does not apply, hence both objectives are not expected to coincide.

Q2: ImageNet is addressed in our general response.

Q3:

We have not yet proven such an extension rigorously, but we are optimistic that an analogous result might hold. The bound in Lemma A.1 extends to general contaminating distributions with finite first moment, since the contamination enters only through ∥m−ν∥−∥ν∥≥−∥m∥\lVert m - \nu \rVert - \lVert \nu \rVert \geq - \lVert m \rVert holding for all ν\nu. The next steps in the proof appear to use the contaminating distribution only through its support and we plan to check those steps carefully to see if we can extend them.

The extension of Theorem 4.2 to other Wasserstein distances follows directly for general 1≤p<∞1 \leq p < \infty since in the proof we characterized the generated distribution as (1−ϵ⋆)δ0+ϵ⋆δb(1-\epsilon^\star)\delta_0 + \epsilon^\star \delta_b, where ϵ∗<ϵ\epsilon^* < \epsilon. Theorem 4.3 also appears to extend to 1≤p<∞1 \leq p < \infty. The only change is in equations (13)-(15) where we can replace the linearity of the expectation with Minkowski’s inequality. Lemma A.2 does not depend on pp and we obtain lim sup⁡R→∞R−1lim sup⁡t↗1Wp(ρtR,P)≤(ϵ∗)1/p\limsup_{R\to\infty} R^{-1}\limsup_{t\nearrow 1} W_p(\rho^R_t,P) \le (\epsilon^*)^{1/p}, while Wp(PRcont,P)/R→ϵ1/pW_p(P^{\mathrm{cont}}_R,P)/R \to \epsilon^{1/p}. So we have a similar improvement. We will update Theorems 4.2 and 4.3 and their proofs accordingly.

Thanks the authors for their responses. However, I believe the comparison between the standard method and the proposed method requires further evaluation. Therefore, I would like to keep my score.

Comments on the paper

1 comment

We thank the reviewers for their helpful comments and suggestions. We first present additional experiments and address two recurring requests: (a) evaluation beyond CIFAR-10 and (b) comparison with other robust losses.

(a) First we tested on a subset of ImageNet. For this we fixed 100 classes of ImageNet at a resolution of 32x32 which is widely used for generative modeling and used this subset for training. We use the same 100 classes as Tian et al. 2020 [ https://arxiv.org/abs/1906.05849 ] for convenience and reproducibility. We reused the same training configuration from our CIFAR-10 experiments with 400,000 updates and only 1 training seed. We are aware that this is only a surrogate for training on full ImageNet with an optimized architecture and the resulting FID values are not optimal. We reused some contamination models from our main study with $\epsilon = 0.10 $ and added, additionally, a small semantic contamination model, where images from 10 different (random) subclasses are considered as contamination. Bold indicates the lowest FID in each row.

ContaminationSquaredL2,1L_{2,1}
Clean11.9311.46
Random noise27.7420.37
Salt-and-pepper13.8212.46
Gaussian blur15.2016.69
Semantic contamination12.8712.05

These results are consistent with our experiments on CIFAR-10. We even improve slightly on the semantic contamination model, where the significance is to be evaluated thoroughly.

(b) We also did a small study for coordinatewise and pseudo-Huber, with the same training configurations as our main experiments. Note that, due to the shortage of time, we only trained on one seed yet. We used the default parameter $ \delta = 1 $ which is also reported in the experiments of Li and Dong (2026). The clean row uses ϵ=0\epsilon = 0 and all corruption rows use ϵ=0.1\epsilon = 0.1. Bold indicates the lowest FID in each row.

ContaminationSquaredL2,1L_{2,1}Coordinatewise HuberPseudo-Huber
Clean3.7383.5133.7413.622
Random noise18.85112.33117.60617.433
Gaussian noise4.7294.1064.6944.497
Salt-and-pepper6.3555.0584.6726.174
Patch noise4.5384.0214.2944.401
Inversion4.7304.3054.8464.529
Gaussian blur6.2967.9986.4826.334
Motion blur6.2577.4016.4306.197

For a velocity residual r∈RDr\in\mathbb{R}^D, coordinatewise Huber uses

\qquad h_1(s)= \begin{cases} \frac12s^2,& |s|\leq 1,\\\\ |s|-\frac12,& |s|>1. \end{cases}$$ Pseudo-Huber instead acts on the RMS residual: $$\ell_{\mathrm{PH}}(r)=\sqrt{1+\frac{\lVert r\rVert_2^2}{D}}-1.$$ Here $D=3\times32\times32$, and both losses are averaged over the training batch. For brevity, we report the intermediate contamination rate $\epsilon = 0.1$. We additionally evaluated $\epsilon = 0.05$ and $\epsilon = 0.15$ and the results are consistent with those in the table above, except that our method also wins for Salt-and-Pepper for \$\\epsilon = 0.05 \$. We only tested one concrete value of \$ \\delta \$ for both and did not optimize over it. As a strength of our method, we emphasize that there is no need to tune a parameter such as \$ \\delta \$. While our method also loses for the Salt-and-Pepper corruption, the general pattern seems to hold that our method performs worse for the blur mechanisms and better for the others.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.