acceptodds
Under review as a conference paper at ICLR 2027

Towards Universal Wasserstein Barycenters through Flow Matching

24 views

Abstract

Defining a weighted mean over probability measures under probability metrics is a central tool in probabilistic machine learning. Under the Wasserstein metric, these are called Wasserstein barycenters. While most approaches compute barycenters for a fixed weight vector, approximating the whole family of barycenters over the simplex, which we call the Wasserstein simplex, remains underexplored. We refer to this problem as Universal Barycenter Approximation, and propose BaryFM, a flow matching model transporting the marginal measures into any barycenter in the Wasserstein simplex. Once trained, the network can draw samples from measures in the Wasserstein simplex through an ordinary differential equation. We validate our method on 4 downstream tasks: domain adaptation, generalization, Bayesian posterior aggregation and algorithmic fairness. BaryFM achieves the best average rank among 15 competing methods across 10 domain adaptation benchmarks, matching or surpassing non-universal solvers.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
optimal transport, wasserstein barycenters, domain adaptation, domain generalization, fairness, bayesian learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3BsROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes BaryFM, a method to learn the whole family of Wasserstein barycenters of a fixed set of distributions with a single model, so that the barycenter for any choice of weights on the simplex can be sampled without retraining. The network has a shared encoder of the sample, the source index and the weights, and two heads. A dual OT head gives an entropic barycentric map, which builds a target for the barycenter with fixed-point steps that start from a weighted average of samples drawn independently from each source. A flow matching head then learns to transport each source to this target, so a new barycenter only needs one ODE solve, and features and labels can be transported together. The contributions are the universal formulation, the two-head architecture and the use of random barycenters for data augmentation. The method is validated on a 2D Swiss roll family with known barycenters, multi-source domain adaptation on ten benchmarks, domain generalization on five benchmarks, Bayesian posterior aggregation and fairness.

Critical strengths
  1. The problem and the design are useful. Most neural barycenter solvers are trained for one weight vector, while BaryFM learns all of them at once with a single network, and sampling a new barycenter then costs one ODE solve. Features and labels are transported together, which is what domain adaptation and augmentation need. This matters for my assessment because it is the main reason the paper would be valuable if the method computes what it claims.
  2. The domain adaptation results are strong and broad. BaryFM has the best average rank over ten benchmarks covering images, EEG and an industrial process, and it beats the strongest baseline (WGF with labels) on eight of them. This is the clearest evidence that the method is useful in practice, even though, as I explain below, it does not show that the learned distributions are barycenters.
Critical weaknesses
  1. The training procedure does not seem to compute the barycenter. In Algorithm 1, every iteration starts again from a weighted average of samples drawn independently from each source, which is the Domain MixUp point, applies one fixed-point step in most experiments, and then updates the dual potential on the result. Unlike WIN or WGF, which keep improving the same barycenter estimate, the estimate here is rebuilt from scratch at every iteration, so the procedure settles where its own target is self-consistent, and I think this point is not the barycenter in general. For example, if all sources are the same standard Gaussian, the barycenter is that Gaussian for every choice of weights, but with exact maps and one fixed-point step a quick calculation shows that the stationary point of Algorithm 1 for three sources has variance about 0.58 instead of 1. More steps make the gap smaller without removing it, and the entropic smoothing adds more shrinkage. This is important because the main claim of the paper is that BaryFM approximates the barycenters across the simplex.

  2. The experiments cannot tell whether the learned distributions are barycenters. The only comparison with known barycenters is the 2D Swiss roll family in Appendix D, where the metric only compares means and covariances, and the number of fixed-point steps, the regularization and the batch size are not reported. With one fixed-point step I would expect Algorithm 1 to be off by around 5 percent on average over this simplex, while Figure 12 reports values around 0.03. So either this experiment uses a different setting from the rest of the paper, or the numbers are not in the units of Eq. 29. The other experiments also do not test the barycenter itself. Domain adaptation and fairness use only uniform weights, and posterior aggregation is scored against the full-data posterior, with the guidance scale and the best weights chosen using that same reference. Without guidance and the semi-unbalanced variant, the BaryFM ranks 7th of 18 in Table 5, and the TDSB baseline is much weaker than in its original paper on the same Wine benchmark (14.22 here against 8.69 in [1]). This I think is important because without a direct test the downstream gains cannot be attributed to barycenters, and the results barely move when the number of fixed-point steps goes from 1 to 100 (Figures 15 and 19), which suggests these tasks are not sensitive to barycenter accuracy.

Overall, I like the problem and the domain adaptation results, but I am not convinced that the method computes what the paper claims, and the experiments are not designed to check it.

[1] Noble, M., De Bortoli, V., Doucet, A., and Durmus, A. Tree-based diffusion Schrödinger bridge with applications to Wasserstein barycenters. NeurIPS 2023.

Questions for authors
  1. Could the authors run BaryFM with several identical sources, where the correct answer is known for every weight, and report the variance of the fixed-point targets and of the flow outputs separately? A good result here would resolve my first weakness.
  2. Which number of fixed-point steps, regularization, batch size and Dirichlet parameter were used in Appendix D, and are Figures 11 and 12 in percent? How does the error at the center of the simplex compare with the error near the vertices?
  3. How were the guidance scale, the k-dropout rate and the best weights in Figure 13 chosen, given that the full posterior is not available in practice?
  4. Were the baselines rerun, and with which tuning budget? Can the authors explain the gap with the published TDSB numbers?
  5. For domain generalization, a baseline that averages samples matched by OT instead of independent ones would show whether the gain comes from the pairing or from the flow model, since the pairing is mostly what separates Domain MixUp from the barycenter.
  6. The paper says only Fan et al. (2020) amortized over the weights, but [2] trains one network for arbitrary weights on grids, [3] learns one velocity field indexed by the simplex, and [4] interpolates labeled datasets for any weights. Positioning BaryFM against these would make the contribution clearer.

[2] Lacombe, J., Digne, J., Courty, N., and Bonneel, N. Learning to generate Wasserstein barycenters. Journal of Mathematical Imaging and Vision, 65, 354–370, 2023. [3] Albergo, M. S., Boffi, N. M., Lindsey, M., and Vanden-Eijnden, E. Multimarginal generative modeling with stochastic interpolants. ICLR 2024. [4] Fan, J. and Alvarez-Melis, D. Generating synthetic datasets by interpolating along generalized geodesics. UAI 2023.

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer 3BsR (1/3)

Dear reviewer,

Thank you for your thoughtful review. We are pleased that you found the problem we are considering and the design to be useful, and that our domain adaptation results are strong and broad. Below you may find answers to the weaknesses you raised, and the questions you have asked.

Weakness 1: CFM targets

Our main claim in the paper is computing an approximate barycenter. There are different sources of bias in our algorithm, as we identified in Table 3. In our revised manuscript, we added a theoretical analysis of these biases, and we show that they are, in general, controlled by,

  • Theorem J.2: The entropic regularization ϵ\epsilon, which trades off the entropic bias (Proposition J.3), decreasing as ϵ→0\epsilon \to 0, against the dual-gap terms (Lemma J.3 and Proposition J.1), which grow as ϵ−1/2\epsilon^{-1/2} and ϵ−1/4\epsilon^{-1/4}.
  • Proposition J.3: Dimensionality of the data, through the entropic bias (the smaller, the more accurate).
  • Lemma J.3, Propositions J.1 and J.2: The optimality gap of both the CFM and Dual OT head (the smaller, the more accurate).
  • Proposition J.4: The residual of the fixed-point iterations (the smaller, the more accurate).

We further added experiments on the analysis of the accuracy of the barycenter targets (Figures 18 and 20).

Weakness 2: On downstream tasks and barycenter accuracy

The experiments cannot tell whether the learned distributions are barycenters. The only comparison with known barycenters is the 2D Swiss roll family in Appendix D, where the metric only compares means and covariances, and the number of fixed-point steps, the regularization and the batch size are not reported. With one fixed-point step I would expect Algorithm 1 to be off by around 5 percent on average over this simplex, while Figure 12 reports values around 0.03.

We added the used hyper-parameters. Essentially, we use LFP=10L_{\text{FP}}=10, ϵ=0.05\epsilon = 0.05, B=1024B=1024, N=10000N=10000, λ∼Dir(α)\lambda \sim \text{Dir}(\alpha), α∼U[0.5,1.0]\alpha \sim \mathcal{U}[0.5, 1.0]. We also added qualitative (Figure 14) and quantitative (Figure 18) analysis on the BaryFM targets, showing their accuracy as a function of LFPL_{\text{FP}}.

The other experiments also do not test the barycenter itself. Domain adaptation and fairness use only uniform weights, and posterior aggregation is scored against the full-data posterior, with the guidance scale and the best weights chosen using that same reference. Without guidance and the semi-unbalanced variant, the BaryFM ranks 7th of 18 in Table 5, and the TDSB baseline is much weaker than in its original paper on the same Wine benchmark (14.22 here against 8.69 in [1]). This I think is important because without a direct test the downstream gains cannot be attributed to barycenters, and the results barely move when the number of fixed-point steps goes from 1 to 100 (Figures 15 and 19), which suggests these tasks are not sensitive to barycenter accuracy.

We refer the reviewer below for answers on BPA. For Domain Adaptation, an interesting finding explaining why the results are insensitive to LFPL_{\text{FP}} comes from the reviewer’s suggestion on experimenting with identical Gaussians. As we grow the dimensionality of the space, we find that the barycenter accuracy becomes robust to LFPL_{\text{FP}}. Since the DA experiments are high dimensional (2048 dimensions for ResNet features, 200 dimensions for CBraMod), it explains the insensitivity to LFPL_{\text{FP}}.

AuthorsOpenReview ↗

Rebuttal to Reviewer 3BsR (2/3)

Question 1: Experiment on identical sources

We added the experiment in Appendix E (Barycenters of Identical Gaussian Measures). We slightly modified the setting to p1=p2=⋯=pK=N(0,Σ)p_{1} = p_{2} = \cdots = p_{K} = \mathcal{N}(0, \Sigma) for a random Σ\Sigma (see appendix for details). This helps avoiding models that rely on N(0,Id)\mathcal{N}(0, Id) having a trivial solution to the problem. We have run this benchmark for dim∈2,16,128\text{dim} \in \\{2, 16, 128\\}.

Here are our findings,

  • BaryFM performs better than NWBU at dimensions 2 and 16, and similarly to it at dimension 128
  • WIN remains more accurate than universal solvers, by a margin that is largest at dimension 16 and smallest at dimension 128
  • The shrinkage effect the reviewer mentioned is indeed correct, but it also affects WIN and NWBU. There are several factors that may induce this shrinkage, among which the use of entropic OT that the reviewer themselves mentioned.
  • Figure 19, last 2 panels: In the comparison, at dimension 128, both BaryFM and WIN shrink the barycenters (BaryFM shrinking much more than WIN), but both manage to better respect the ground-truth shape, and so they end up having a better barycenter accuracy.
  • Figure 20: BaryFM targets can be broken down into the points produced by the FP procedure and the sampled CFM pairs (z0,z1)(z_0, z_1) described between lines 246 and 256. In general, more iterations of the FP procedure helps with barycenter accuracy in low dimensions, while in high dimensions, accuracy is insensitive to it. This finding helps explaining why the Domain Adaptation results are robust to L_FP and it indeed justifies using L_FP = 1.

Question 2: Hyper-parameters of the Swiss Roll experiment

We added the complete set of hyper-parameters used in Section D: LFP=10L_{\text{FP}} = 10, ϵ=0.05\epsilon = 0.05, B=1024B = 1024, N=10000N=10000, λ∼Dir(α)\lambda \sim \text{Dir}(\alpha), α∼U[0.5,1.0]\alpha \sim \mathcal{U}[0.5, 1.0] at each iteration. All methods are evaluated on ngen=10000n_{\text{gen}} = 10000 samples.

Question 3 and 4: Hyper-parameter selection, and the gap to TDSB published results in Bayesian Posterior Aggregation experiments

Hyper-parameter selection. In the current Wasserstein Barycenter literature (Fan et al., 2020; Korotin et al., 2021; Noble et al., 2023; Howard et al., 2025), methods are scored against samples from the full posterior q0q_0 and reported with fixed hyper-parameters. There is currently no hyper-parameter selection criterion on this task, on this literature.

We scored methods in the same way. BaryFM hyper-parameters were selected using BW2-UVP against q0q_0, which includes the guidance scale ss, and the dropout rate. To ensure fairness on the comparison against other methods, we applied the same rule for all tested methods.

With that said, we agree with the reviewer that a selection rule that does not depend on the ground truth q0q_0 is needed. However, we consider the design of such rule to be outside the scope of our paper. Indeed, BPA represents 1 out of 4 downstream tasks we consider, and its results are meant to illustrate what universality and guidance make possible. Therefore, we softened the state-of-the-art claims in the revised version of our manuscript.

Gap to TDSB reported values. The gap comes from the evaluation protocol. (Noble et al., 2023; Appendix G.2) reports,

For each of the three settings, we keep the best result among the 30 IPF iterations. In this setting, TreeDSB and crWB have roughly the same training time.

Inspecting closely their code shows that, at each IPF iteration, TDSB is evaluated on the samples transported from a single marginal. Hence, the reported value is the best BW$_2$-UVP against the ground-truth posterior among 30 evaluations, each of which corresponds to 1 iterate and 1 source marginal.

As stated in our BPA section, we score all models in a uniform way. In other words, we flow all marginals into the barycenter, yielding 3×9900=297003 \times 9900 = 29700 points, and we report the last iterate. We consider this comparison fairer, especially to barycenter methods that do not learn per-marginal flows.

AuthorsOpenReview ↗

Rebuttal to Reviewer 3BsR (3/3)

Question 5: New baselines for DG

Thank you for that suggestion, this is indeed a good baseline for DG. We went ahead and tested the Generalized Geodesics on Office 31, Office Home, BCI-IV, and SEED-VIG (cf. tables in Section I.3) which corresponds to that strategy, with the labels taken into account in the ground-cost. This approach ranks 5th, 5th, 3rd and 2nd, respectively, showing that it is a strong baseline on EEG, and a weak one in images. For comparison, BaryFM$_{\text{random}}$ ranks 1st, 1st, 2nd and 1st on each of these benchmarks, thus being consistently a stronger method.

Question 6: Positioning against prior work

Indeed, all of these references are relevant and closely related to our work. Thank you for pointing those out. However, we consider that they solve different, but related problems to ours.

  • On the precedent of Lacombe et al., [2]: Their approach does learn to generate barycenters with arbitrary weights in the simplex. There is an important distinction, however, especially with the kind of barycenters our BaryFM and NWBU computes. Lacombe et al.,, use histogram measures (e.g., an image, normalized to sum to 1), while ours is the “free-support” kind of barycenters in the sense of (Cuturi and Doucet, 2014).
  • Multi-marginal stochastic interpolants [3]: This work solves a different but related problem to Wasserstein Barycenters, called Multi-Marginal Optimal Transport. Indeed, the problems coincide for a specific choice of ground-cost (see, for instance, the discussion we added in Appendix B), but are, in general, different.
  • Positioning against Generalized Geodesics [4]: These authors interpolate labeled datasets under the OTDD geometry [5]. While related to the downstream tasks we considered in the paper (Domain Adaptation and Generalization), their method computes interpolations along generalized geodesics (see (Agueh and Carlier, 2011), especially their Definition 7.4). As a result, these authors are computing a different object than ours.

We have added this discussion in our Appendix B, related work, and we have corrected a few claims in the introduction, especially related to Lacombe et al.

Reviewer_ue67OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to approximate Wasserstein barycenters of a fixed collection of input distributions for arbitrary weights. The authors propose BaryFM, which consists of a shared encoder and two task-specific heads. A dual optimal transport head estimates transport potentials used to construct approximate barycenters through fixed-point iterations and to obtain transport couplings for training, while a conditional flow matching head learns to transport samples from the input distributions to the corresponding approximate barycenters. Experiments cover domain adaptation, domain generalization, Bayesian posterior aggregation, and algorithmic fairness.

Critical strengths
  1. The method addresses a computational need. When multiple barycentric weight vectors need to be queried, retraining a solver for each weight vector incurs additional computational cost. BaryFM uses a single trained model to approximate Wasserstein barycenters (WB) for different weight vectors.

  2. The experimental evaluation is broad. In the domain adaptation experiments cover 10 benchmarks and compare BaryFM with 15 methods, with BaryFM achieving the best average rank.

Critical weaknesses
  1. The analysis of barycenter approximation errors remains limited. Appendix D primarily uses BW2-UVP% to compare the results with analytically known reference barycenters. This metric captures only differences in means and covariances, providing limited evidence of full distributional accuracy for non-Gaussian barycenters. Furthermore, the paper does not clearly quantify fixed-point target construction errors and subsequent generation errors separately.
  2. Unclear efficiency gains from the dual OT head. With neural dual estimation, target construction remains quadratic in batch size. The paper lacks a comparison of end-to-end training time and peak GPU memory between BaryFM and BaryFM-Emp at comparable barycenter approximation accuracy, making it difficult to assess how much this computational simplification improves overall training efficiency.
  3. Flow matching do not give OT transport solutions, re-flow will decrease the Wasserstein distance but it is not involved in this paper. Thus, I cannot see what makes applying flow matching in WB appropriate.
Questions for authors
  1. How accurately do the learned OT couplings approximate the reference solutions obtained using a converged Sinkhorn solver, in terms of marginal constraint violations and transport costs?

  2. For a fixed (\lambda), how consistent are the distributions generated from different source marginals?

  3. Could the authors specify the values of (\alpha,\beta,\rho) used in the experiments and explain how they were selected?

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer ue67

Dear Reviewer,

Thank you for your thoughtful review. We are pleased that you found that our method addresses a computational need, and that our experimental evaluation is broad. Below you may find answers to the weaknesses you raised, and the questions you have asked.

Weakness 1: Analysis of Barycenter Approximation Errors

The reviewer wrote a few points, which we answer in the following,

Appendix D primarily uses BW2-UVP% to compare the results with analytically known reference barycenters. This metric captures only differences in means and covariances, providing limited evidence of full distributional accuracy for non-Gaussian barycenters

Following the reviewer’s suggestion, we have added the 2-Wasserstein distance in our comparison. We refer the reviewer to Figure 12 in our appendix.

Furthermore, the paper does not clearly quantify fixed-point target construction errors and subsequent generation errors separately.

We now report these separately. Figure 18 (Swiss roll) and Figure 20 (identical Gaussians) show the accuracy of the fixed-point samples, of the CFM targets and of the BaryFM samples as a function of LFPL_{\text{FP}}.

Weakness 2: Efficiency gains in comparison with the empirical version of our algorithm

Indeed, our initial manuscript lacked a comparison in the running times of BaryFM and BaryFM-emp. The reviewer may find a complete running time comparison, both in terms of sampling time and training time, of the different tested methods on the Swiss Roll benchmark (Figure 13), and a comparison of BaryFM and BaryFM-Emp in training time, peak GPU memory and barycenter accuracy (Figure 16). Overall, BaryFM trains significantly faster than its empirical version (227s vs. 2647s), mainly due to the Sinkhorn loop in the empirical case, at a comparable accuracy (BW2-UVP% of 0.030 against 0.035, W2W_2 of 0.064 for both). BaryFM has a higher peak GPU memory (186 MiB against 162 MiB) because of the dual OT head.

Weakness 3: Flow Matching and Wasserstein Barycenters

Flow matching do not give OT transport solutions, re-flow will decrease the Wasserstein distance but it is not involved in this paper

Indeed, the learned field in CFM does not, in general, give the OT map between its endpoints. In our method, CFM acts as a generative model for the approximated regularized barycenters, that is, a way to sample from these measures. We do, however, approximate the OT maps via the entropic map defined in equation (9), and now analyzed in Sections J.1 and J.3, in the appendix. While reflow could be applied on top of BaryFM, we did not use it in our experiments or framework.

Thus, I cannot see what makes applying flow matching in WB appropriate.

We chose CFM for 3 reasons. First, the accuracy of the generated measure can be controlled by the CFM optimality gap (Proposition J.2), independently of the flow being an OT map. Second, our CFM strategy does not sample (z0,z1)(z_0, z_1) independently. Indeed, we adopt the strategy of Tong et al. (2023) and sample the endpoints via a joint law of these variables. Third, and most importantly, CFM makes it simple to acquire new samples from measures in W(p1:K)\mathcal{W}(p_{1:K}), since, with a single ODE solve, it transports features and labels jointly, as domain adaptation and generalization require, and admits classifier-free guidance, which we use in Bayesian posterior aggregation.

Answer to Questions

Question 1: How accurate are the learned OT couplings

We refer the reviewer to Proposition J.1 in the revised manuscript. Briefly, a sub-optimal potential ff induces a transport plan γ~f\tilde{\gamma}_{f} which has as left marginal pp, and right marginal an induced q~f\tilde{q}_f (See Eq. 43 for its density). We prove a few bounds on the marginal violation, meaning with respect the total variation TV(q,q~f)\text{TV}(q, \tilde{q}_f), the 1-Wasserstein distance and the 2-Wasserstein distance, all of which are bounded by the dual optimality gap.

Question 2: How consistent are distributions generated from different source marginals?

We refer the reviewer to Section J.4, where we prove a bound on the W2 of the flow from different sources in Corollary J.1. In brief, the consistency between multiple sources is bounded by the CFM and dual optimality gaps.

Question 3: Values of α\alpha, β\beta and ρ\rho

β\beta is used in Domain Adaptation and Generalization exclusively. It is automatically set within mini-batches, as β=max⁡i≠j∥xi−xj∥22\beta = \max_{i \neq j} \lVert x_{i} - x_{j} \rVert_{2}^{2}. This means that matching samples from different classes is as costly as the most far apart points within that mini-batch. We added this information to Section H.2.

ρ\rho is used in Domain Generalization exclusively, and is set to 0.5 for all methods.

α\alpha is used in all experiments. We have reported its values in Tables 6, 11 and 28.

Dear Reviewer,

Thank you again for the careful reading of our manuscript. Your remarks have helped improving the quality of our manuscript.

As the discussion period comes to a close, we would appreciate it if you could let us know whether our response and the revised manuscript address your concerns. We would gladly clarify any remaining point. If these changes resolve your concerns, we would be grateful if you could update your assessment accordingly.

Reviewer_MEkkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new algorithm for computing the barycenters in the Wasserstein simplex, i.e., the method for approximating Wasserstein barycenters for a random set of weights. The method is tested on several downstream tasks.

Critical strengths

The paper is quite well-written and structured. A lot of work has been done in terms of testing the method on downstream tasks.

Critical weaknesses

[W1] The developed approach is aimed to approximate the Wasserstein barycenters for arbitrary sets of weights. However, the construction of the method raises concerns regarding the quality of this approximation. Indeed, the method is composed of several stages.

Firstly, the authors randomly initialize the weights and set of points from barycenter distribution; then consider discrete OT problem between samples of input distributions and current barycenter; use current approximations of OT potentials to update the discrete approximation of barycenter distribution. Importantly, to find this new approximation, the authors consider barycentric projections of OT plans between discrete approximations of the underlying distributions. These projections will likely introduce bias in the solutions in comparison to the true OT maps between the given measures.

Secondly, the current approximation of barycenter distributions is used to learn the approximations of OT potentials which are actually learned as neural networks approximating the solutions of discrete OT problem between the available set of points from input distributions and current “barycenter”. It is an open question, how well these approximations built upon discrete solutions approximate the actual solutions between continuous distributions.

Thirdly, current approximations of potentials and barycenter points are used to construct the pairs for learning the conditional FM model between the input distributions and barycenter.

Thus, there are a lot of steps which might introduce biases in the solution of the stated universal barycenter approximation problem. Thus, it is expected that the paper at least provides some theoretical result supporting the idea that the method indeed provides good approximations of the barycenter for the considered problem.

[W2] This weakness is related to the previous one. The experimental section includes only one experiment on Swiss roll measures with the known ground-truth solutions. This experiment is very important for showing if the developed approach allows for obtaining good approximations for barycenter problem solutions. Overall, I do agree with this experimental setup, however, the choice of baseline approaches is very limited. I am wondering why the OT barycenter baselines, e.g., CW2B, NOT, U-NOT, WIN) which are included in many other experiments from this paper (see Tables 8, 16, 17, 18, 23, 24, 25) are omitted here? I kindly suggest the authors include them in the comparison.

[W3] The semi-unbalanced BaryFM model is described in Appendix С.3 but is not tested in most of the experimental settings including Swiss roll, fairness, domain adaptation and domain generalization. I kindly suggest the authors include some of the additional experiments during the rebuttal phase.

[W4] In line 1287, check the spaces, texts coincide.

In summary, there are no theoretical results supporting the idea that the method indeed allows for obtaining good approximations of barycenters and limited experimental justification of this fact.

Questions for authors

See Weaknesses section.

1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer MEkk

Dear Reviewer,

Thank you for your thoughtful review. We are pleased that you found our paper to be quite well-structured and well written, and that you recognized the effort that went through the preparation of the downstream tasks. We also thank you for pointing out the colliding text in our appendix. This issue has been corrected in the new version of our manuscript.

Below, you may find clarifications about the weaknesses you raised.

Weakness 1: Theoretical Analysis

Indeed, further theoretical justification of the method and its accuracy are needed. We refer the reviewer to Appendix J in our revised manuscript, which contains answers to the questions you raised, especially, we quantify:

  • In Proposition J.1, the error in the marginal of the entropic plan γ~f\tilde{\gamma}_f induced by an arbitrary potential ff.
  • In Proposition J.3, the bias in the transport maps introduced by the entropic barycentric projection.
  • In Proposition J.4, the closeness in W2-distance of the constructed targets qλ(LFP)q_{\lambda}^{(L_{\text{FP}})} and the barycenter

Then, Theorem J.2 ties together these results in a bound on the accuracy of the samples generated by CFM for each marginal. Corollary J.1 bounds the cross-source consistency (flowing sampling from different marginals to the barycenter), and Corollary J.2 provides a bound for the inference of BaryFM.

Weakness 2: Baselines in the Swiss Roll Experiment

Thank you for your suggestion. Following your advice, we included several baselines in the Swiss Roll experiments: UNOT, NOT, WGF, WIN, W2B, TSBM, NWBU and NF. We show both qualitative (Figures 11, 14, and 15) and quantitative (Figures 12, 13, 16 and 17) comparisons, including new studies on the comparison of BaryFM-U with UNOT (see next response).

Weakness 3: Additional Experiments with the Unbalanced version

We have tested the unbalanced version on Domain Adaptation (see Figure 29), Swiss Roll (Figures 15 and 17) and Fairness (Figure 27),

  • In Domain Adaptation, gains are not significant over most datasets or there is a negative impact over the obtained barycenters.
  • On the Swiss Roll benchmark, adding the unbalanced penalty tends to deform barycenters at low penalties, and at high penalties there is no qualitative difference to the balanced version. We verified the same effect for UNOT.
  • In Fairness, unbalanced barycenters are close to the no repair baseline, which is similar to our findings on UNOT. Again, we test both methods for a varying penalty τ\tau, achieving similar results.

So, overall, the only benchmark where unbalanced barycenters (BaryFM-U and UNOT) seem to be favored is Bayesian Posterior Aggregation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2)

We thank the reviewers for their careful reading. All changes in our revised manuscript are highlighted in blue.

Summary of Positive Points

We are glad that the reviewers found the problem and the design of BaryFM useful, since a single network approximates barycenters for every weight in the simplex, and a new barycenter only costs 1 ODE solve (Reviewers ue67 and 3BsR).

  • Downstream tasks. The reviewers found the experimental evaluation broad (Reviewers MEkk and ue67), and the domain adaptation results strong, with the best average rank over 10 benchmarks covering images, EEG and an industrial process (Reviewers ue67 and 3BsR).
  • Joint measures. Transporting features and labels together is what domain adaptation and augmentation require (Reviewer 3BsR).
  • Presentation. Reviewer MEkk found the paper well written and structured.

Summary of Negative Points

Below, we identify 3 overarching criticisms on our initial manuscript.

Lack of theoretical analysis (MEkk W1). We added Section J, which bounds the distance between the measures generated by BaryFM and the barycenter qλ⋆q^\star_\lambda in terms of the optimality gaps of the 2 heads, the residual of the fixed-point iterations and the entropic regularization ϵ\epsilon. The results follow the stages of Algorithm 1 listed by Reviewer MEkk.

  • Dual head. Lemma J.3 bounds the error of the neural entropic map by the dual optimality gap ED\mathcal{E}_D, and Proposition J.1 bounds the marginal violation of the coupling used to sample the CFM pairs (Reviewer ue67, Q1).
  • Entropic maps. Proposition J.3 bounds the bias of the entropic map with respect to the Monge map, which is of order O(ϵlog⁡(1/ϵ))O(\sqrt{\epsilon\log(1/\epsilon)}).
  • Flow matching. Proposition J.2 bounds the distance between the flow endpoint and its target by the CFM gap EC\mathcal{E}_C.
  • Fixed-point outputs. Proposition J.4 bounds the Wasserstein gradient of the barycenter functional at the output of the fixed-point iterations and, under an error bound condition (Assumption 5), its distance to qλ⋆q^\star_\lambda.
  • BaryFM. Theorem J.2 combines these results into a bound for the flow endpoint of each source, Corollary J.1 bounds the cross-source consistency (Reviewer ue67, Q2), and Corollary J.2 bounds the accuracy of the pooled BaryFM samples.

Barycenter accuracy (MEkk W2, ue67 W1, 3BsR W2). We rewrote the Swiss roll experiment (Section D) and we now state its hyper-parameters (LFP=10L_{\text{FP}} = 10, ϵ=0.05\epsilon = 0.05, B=1024B = 1024, α∼U[0.5,1]\alpha \sim \mathcal{U}[0.5, 1] for each λ\lambda).

  • More baselines. Following Reviewer MEkk, we compare 9 solvers: BaryFM, NWBU, WGF, CW2B, NOT, U-NOT, WIN, TSBM and NF (Figures 11 and 12).
  • Metrics. Following Reviewers ue67 and 3BsR, we report W2W_2 alongside BW2-UVP%, together with the sampling floor, that is, each metric between 2 independent draws of the ground-truth (Figure 12). BaryFM has the lowest W2W_2 among the 9 solvers (0.064) and a BW2-UVP% of 0.030.
  • Coverage of the simplex. Figures 11, 14 and 15 show the generated barycenters against the ground-truth on the 3 vertices, the 3 edge midpoints and the center of the simplex, both for solvers refit per weight (e.g., NOT, U-NOT) and for universal solvers. Figures 12, 17 and 18 report the metrics aggregated over the same 7 weights. For identical Gaussian measures, Figure 19 reports the vertices separately.
  • Targets and generated samples. Figure 18 (Swiss roll) and Figure 20 (identical Gaussians) report separately the accuracy of the fixed-point samples, of the CFM targets and of the BaryFM samples as a function of LFPL_{\text{FP}}.
  • Dual head against empirical OT. Figure 14 (qualitative) and Figure 16 compare BaryFM with BaryFM-Emp, the version of our method without the dual estimation head. For LFP=10L_{\text{FP}} = 10, training takes 227 s against 2647 s, that is, the dual head makes training 11.7 times faster, at a comparable accuracy (BW2-UVP% of 0.030 against 0.035, W2W_2 of 0.064 for both) and peak GPU memory (186 MiB against 162 MiB).
  • Running time. Figure 13 reports training and inference times. For the 7 weights of Figure 11, BaryFM takes 231 s in total, while solvers refit per weight take between 0.3 h and 5.2 h.
AuthorsOpenReview ↗

General Response (2/2)

Bias of the fixed-point targets (3BsR W1 and Q1). We ran the experiment suggested by Reviewer 3BsR, with K=3K = 3 identical Gaussian measures in dimensions 2, 16 and 128 (Section E). Figure 20 reports the variance of the fixed-point samples, of the CFM targets and of the flow outputs separately. The experiment confirms the shrinkage pointed out by the reviewer. For instance, in dimension 2, the targets retain approximately 74% of the variance for LFP=1L_{\text{FP}} = 1 and approximately 92% for LFP=10L_{\text{FP}} = 10. In dimension 128, the retained variance is approximately 62% for the targets and approximately 75% for BaryFM, and it does not improve with LFPL_{\text{FP}}. As we argue below, this finding helps explaining why Domain Adaptation results are robust to the values of LFPL_{\text{FP}} and hence justify why we use LFP=1L_{\text{FP}} = 1. We further compare 3 initialization strategies (Section C.5, Figures 14 and 19).

Other additions. Following Reviewer MEkk, semi-unbalanced BaryFM is now evaluated on the Swiss roll (Figures 15 and 17), on fairness (Figure 27) and on 8 domain adaptation benchmarks (Figure 29). Following Reviewer 3BsR, we position BaryFM against Lacombe et al. (2023), Albergo et al. (2024) and Fan & Alvarez-Melis (2023) in Section B, and we add the generalized geodesics of Fan & Alvarez-Melis (2023) as a domain generalization baseline (Tables 29, 30, 32 and 33). We report the range of the Dirichlet parameter, α∈[0.5,1]\alpha \in [0.5, 1] (Tables 6, 11 and 28), and ρ=0.5\rho = 0.5 (Section I.2).

References

[2] Lacombe, J., Digne, J., Courty, N., and Bonneel, N. Learning to generate Wasserstein barycenters. Journal of Mathematical Imaging and Vision, 65, 354–370, 2023.

[3] Albergo, M. S., Boffi, N. M., Lindsey, M., and Vanden-Eijnden, E. Multimarginal generative modeling with stochastic interpolants. ICLR 2024.

[4] Fan, J. and Alvarez-Melis, D. Generating synthetic datasets by interpolating along generalized geodesics. UAI 2023.

[5] Alvarez-Melis, David, and Nicolo Fusi. "Geometric dataset distances via optimal transport." Advances in Neural Information Processing Systems 33 (2020): 21428-21439.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.