- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes BaryFM, a method to learn the whole family of Wasserstein barycenters of a fixed set of distributions with a single model, so that the barycenter for any choice of weights on the simplex can be sampled without retraining. The network has a shared encoder of the sample, the source index and the weights, and two heads. A dual OT head gives an entropic barycentric map, which builds a target for the barycenter with fixed-point steps that start from a weighted average of samples drawn independently from each source. A flow matching head then learns to transport each source to this target, so a new barycenter only needs one ODE solve, and features and labels can be transported together. The contributions are the universal formulation, the two-head architecture and the use of random barycenters for data augmentation. The method is validated on a 2D Swiss roll family with known barycenters, multi-source domain adaptation on ten benchmarks, domain generalization on five benchmarks, Bayesian posterior aggregation and fairness.
Critical strengths
- The problem and the design are useful. Most neural barycenter solvers are trained for one weight vector, while BaryFM learns all of them at once with a single network, and sampling a new barycenter then costs one ODE solve. Features and labels are transported together, which is what domain adaptation and augmentation need. This matters for my assessment because it is the main reason the paper would be valuable if the method computes what it claims.
- The domain adaptation results are strong and broad. BaryFM has the best average rank over ten benchmarks covering images, EEG and an industrial process, and it beats the strongest baseline (WGF with labels) on eight of them. This is the clearest evidence that the method is useful in practice, even though, as I explain below, it does not show that the learned distributions are barycenters.
Critical weaknesses
-
The training procedure does not seem to compute the barycenter. In Algorithm 1, every iteration starts again from a weighted average of samples drawn independently from each source, which is the Domain MixUp point, applies one fixed-point step in most experiments, and then updates the dual potential on the result. Unlike WIN or WGF, which keep improving the same barycenter estimate, the estimate here is rebuilt from scratch at every iteration, so the procedure settles where its own target is self-consistent, and I think this point is not the barycenter in general. For example, if all sources are the same standard Gaussian, the barycenter is that Gaussian for every choice of weights, but with exact maps and one fixed-point step a quick calculation shows that the stationary point of Algorithm 1 for three sources has variance about 0.58 instead of 1. More steps make the gap smaller without removing it, and the entropic smoothing adds more shrinkage. This is important because the main claim of the paper is that BaryFM approximates the barycenters across the simplex.
-
The experiments cannot tell whether the learned distributions are barycenters. The only comparison with known barycenters is the 2D Swiss roll family in Appendix D, where the metric only compares means and covariances, and the number of fixed-point steps, the regularization and the batch size are not reported. With one fixed-point step I would expect Algorithm 1 to be off by around 5 percent on average over this simplex, while Figure 12 reports values around 0.03. So either this experiment uses a different setting from the rest of the paper, or the numbers are not in the units of Eq. 29. The other experiments also do not test the barycenter itself. Domain adaptation and fairness use only uniform weights, and posterior aggregation is scored against the full-data posterior, with the guidance scale and the best weights chosen using that same reference. Without guidance and the semi-unbalanced variant, the BaryFM ranks 7th of 18 in Table 5, and the TDSB baseline is much weaker than in its original paper on the same Wine benchmark (14.22 here against 8.69 in [1]). This I think is important because without a direct test the downstream gains cannot be attributed to barycenters, and the results barely move when the number of fixed-point steps goes from 1 to 100 (Figures 15 and 19), which suggests these tasks are not sensitive to barycenter accuracy.
Overall, I like the problem and the domain adaptation results, but I am not convinced that the method computes what the paper claims, and the experiments are not designed to check it.
[1] Noble, M., De Bortoli, V., Doucet, A., and Durmus, A. Tree-based diffusion Schrödinger bridge with applications to Wasserstein barycenters. NeurIPS 2023.
Questions for authors
- Could the authors run BaryFM with several identical sources, where the correct answer is known for every weight, and report the variance of the fixed-point targets and of the flow outputs separately? A good result here would resolve my first weakness.
- Which number of fixed-point steps, regularization, batch size and Dirichlet parameter were used in Appendix D, and are Figures 11 and 12 in percent? How does the error at the center of the simplex compare with the error near the vertices?
- How were the guidance scale, the k-dropout rate and the best weights in Figure 13 chosen, given that the full posterior is not available in practice?
- Were the baselines rerun, and with which tuning budget? Can the authors explain the gap with the published TDSB numbers?
- For domain generalization, a baseline that averages samples matched by OT instead of independent ones would show whether the gain comes from the pairing or from the flow model, since the pairing is mostly what separates Domain MixUp from the barycenter.
- The paper says only Fan et al. (2020) amortized over the weights, but [2] trains one network for arbitrary weights on grids, [3] learns one velocity field indexed by the simplex, and [4] interpolates labeled datasets for any weights. Positioning BaryFM against these would make the contribution clearer.
[2] Lacombe, J., Digne, J., Courty, N., and Bonneel, N. Learning to generate Wasserstein barycenters. Journal of Mathematical Imaging and Vision, 65, 354–370, 2023. [3] Albergo, M. S., Boffi, N. M., Lindsey, M., and Vanden-Eijnden, E. Multimarginal generative modeling with stochastic interpolants. ICLR 2024. [4] Fan, J. and Alvarez-Melis, D. Generating synthetic datasets by interpolating along generalized geodesics. UAI 2023.
5 comments
Rebuttal to Reviewer 3BsR (2/3)
Question 1: Experiment on identical sources
We added the experiment in Appendix E (Barycenters of Identical Gaussian Measures). We slightly modified the setting to for a random (see appendix for details). This helps avoiding models that rely on having a trivial solution to the problem. We have run this benchmark for .
Here are our findings,
- BaryFM performs better than NWBU at dimensions 2 and 16, and similarly to it at dimension 128
- WIN remains more accurate than universal solvers, by a margin that is largest at dimension 16 and smallest at dimension 128
- The shrinkage effect the reviewer mentioned is indeed correct, but it also affects WIN and NWBU. There are several factors that may induce this shrinkage, among which the use of entropic OT that the reviewer themselves mentioned.
- Figure 19, last 2 panels: In the comparison, at dimension 128, both BaryFM and WIN shrink the barycenters (BaryFM shrinking much more than WIN), but both manage to better respect the ground-truth shape, and so they end up having a better barycenter accuracy.
- Figure 20: BaryFM targets can be broken down into the points produced by the FP procedure and the sampled CFM pairs described between lines 246 and 256. In general, more iterations of the FP procedure helps with barycenter accuracy in low dimensions, while in high dimensions, accuracy is insensitive to it. This finding helps explaining why the Domain Adaptation results are robust to L_FP and it indeed justifies using L_FP = 1.
Question 2: Hyper-parameters of the Swiss Roll experiment
We added the complete set of hyper-parameters used in Section D: , , , , , at each iteration. All methods are evaluated on samples.
Question 3 and 4: Hyper-parameter selection, and the gap to TDSB published results in Bayesian Posterior Aggregation experiments
Hyper-parameter selection. In the current Wasserstein Barycenter literature (Fan et al., 2020; Korotin et al., 2021; Noble et al., 2023; Howard et al., 2025), methods are scored against samples from the full posterior and reported with fixed hyper-parameters. There is currently no hyper-parameter selection criterion on this task, on this literature.
We scored methods in the same way. BaryFM hyper-parameters were selected using BW2-UVP against , which includes the guidance scale , and the dropout rate. To ensure fairness on the comparison against other methods, we applied the same rule for all tested methods.
With that said, we agree with the reviewer that a selection rule that does not depend on the ground truth is needed. However, we consider the design of such rule to be outside the scope of our paper. Indeed, BPA represents 1 out of 4 downstream tasks we consider, and its results are meant to illustrate what universality and guidance make possible. Therefore, we softened the state-of-the-art claims in the revised version of our manuscript.
Gap to TDSB reported values. The gap comes from the evaluation protocol. (Noble et al., 2023; Appendix G.2) reports,
For each of the three settings, we keep the best result among the 30 IPF iterations. In this setting, TreeDSB and crWB have roughly the same training time.
Inspecting closely their code shows that, at each IPF iteration, TDSB is evaluated on the samples transported from a single marginal. Hence, the reported value is the best BW$_2$-UVP against the ground-truth posterior among 30 evaluations, each of which corresponds to 1 iterate and 1 source marginal.
As stated in our BPA section, we score all models in a uniform way. In other words, we flow all marginals into the barycenter, yielding points, and we report the last iterate. We consider this comparison fairer, especially to barycenter methods that do not learn per-marginal flows.
Rebuttal to Reviewer 3BsR (3/3)
Question 5: New baselines for DG
Thank you for that suggestion, this is indeed a good baseline for DG. We went ahead and tested the Generalized Geodesics on Office 31, Office Home, BCI-IV, and SEED-VIG (cf. tables in Section I.3) which corresponds to that strategy, with the labels taken into account in the ground-cost. This approach ranks 5th, 5th, 3rd and 2nd, respectively, showing that it is a strong baseline on EEG, and a weak one in images. For comparison, BaryFM$_{\text{random}}$ ranks 1st, 1st, 2nd and 1st on each of these benchmarks, thus being consistently a stronger method.
Question 6: Positioning against prior work
Indeed, all of these references are relevant and closely related to our work. Thank you for pointing those out. However, we consider that they solve different, but related problems to ours.
- On the precedent of Lacombe et al., [2]: Their approach does learn to generate barycenters with arbitrary weights in the simplex. There is an important distinction, however, especially with the kind of barycenters our BaryFM and NWBU computes. Lacombe et al.,, use histogram measures (e.g., an image, normalized to sum to 1), while ours is the “free-support” kind of barycenters in the sense of (Cuturi and Doucet, 2014).
- Multi-marginal stochastic interpolants [3]: This work solves a different but related problem to Wasserstein Barycenters, called Multi-Marginal Optimal Transport. Indeed, the problems coincide for a specific choice of ground-cost (see, for instance, the discussion we added in Appendix B), but are, in general, different.
- Positioning against Generalized Geodesics [4]: These authors interpolate labeled datasets under the OTDD geometry [5]. While related to the downstream tasks we considered in the paper (Domain Adaptation and Generalization), their method computes interpolations along generalized geodesics (see (Agueh and Carlier, 2011), especially their Definition 7.4). As a result, these authors are computing a different object than ours.
We have added this discussion in our Appendix B, related work, and we have corrected a few claims in the introduction, especially related to Lacombe et al.
Rebuttal to Reviewer 3BsR (1/3)
Dear reviewer,
Thank you for your thoughtful review. We are pleased that you found the problem we are considering and the design to be useful, and that our domain adaptation results are strong and broad. Below you may find answers to the weaknesses you raised, and the questions you have asked.
Weakness 1: CFM targets
Our main claim in the paper is computing an approximate barycenter. There are different sources of bias in our algorithm, as we identified in Table 3. In our revised manuscript, we added a theoretical analysis of these biases, and we show that they are, in general, controlled by,
We further added experiments on the analysis of the accuracy of the barycenter targets (Figures 18 and 20).
Weakness 2: On downstream tasks and barycenter accuracy
We added the used hyper-parameters. Essentially, we use LFP=10, ϵ=0.05, B=1024, N=10000, λ∼Dir(α), α∼U[0.5,1.0]. We also added qualitative (Figure 14) and quantitative (Figure 18) analysis on the BaryFM targets, showing their accuracy as a function of LFP.
We refer the reviewer below for answers on BPA. For Domain Adaptation, an interesting finding explaining why the results are insensitive to LFP comes from the reviewer’s suggestion on experimenting with identical Gaussians. As we grow the dimensionality of the space, we find that the barycenter accuracy becomes robust to LFP. Since the DA experiments are high dimensional (2048 dimensions for ResNet features, 200 dimensions for CBraMod), it explains the insensitivity to LFP.