acceptodds
Under review as a conference paper at ICLR 2027

Vertical Sliced Wasserstein Distances for Meta-Measures

Abstract

Many modern learning problems require comparing probability distributions whose individual observations are themselves probability measures, naturally giving rise to *meta-measures* in . However, optimal transport distances on this space can be prohibitively expensive to evaluate. We introduce the *vertical sliced Wasserstein* (VSW) distance, a scalable metric on . Rather than slicing along the transport geometry of , VSW projects each inner measure using continuous test functions and compares the resulting one-dimensional distributions with one-dimensional Wasserstein distance. We prove that VSW is a valid metric on , metrizes weak convergence, and admits dimension-independent statistical convergence guarantees. Empirically, VSW provides high accuracy and efficiency compared with existing distances for meta-measures. We further derive a practical particle gradient for the empirical objective and demonstrate its use for gradient-based transport of meta-measures. VSW provides a simple and scalable alternative for comparing and optimizing distributions of distributions.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Optimal Transport, Sliced Wasserstein, Meta-measures, Wasserstein over Wasserstein, Dataset Distances

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_kDCNOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Hierarchical WoW distance compute distance between measures over measures. While they can be computed, this is slow. Previous works have proposed slicing procedure as a workaround. The paper proposes a new slicing route via embedding measures in a Hilbert space and to use projections in that space.

Critical strengths

I think that this is a very nice paper, which considers the efficient computation of hierarchical OT distances. These distances and their computation have recently attracted attention and the proposed distance provides a very interesting and new direction. This is a clear and well-written paper based on very well-rounded theory and comparison with other works. In particular, for proposed distance we again get a well-behaved sample complexity which is useful for a lot of applications.

Given my overall impression, I would vote for acceptance. However, I still have some remaining concerns (see below). While I currently lean toward acceptance, my final recommendation will depend on how convincingly the authors address these concerns in their rebuttal.

Critical weaknesses

The 8-Gaussians and checkerboard experiment (Figure 7) is insufficiently visualized and evaluated. Basically, each checkerboard square is one measure and each Gaussian its own measure, i.e., a hierarchy of measures. Thus, the visualization should use color-coded point clouds to make clear that measures not mix and in the plot it should track the WoW distance over time, not the Euclidean Wasserstein distance. I think that the current presentation and evalution can be misleading for a reader.

The bandwidth is chosen pair-dependent in the OTDD (Figures 2–3), point clouds (Figure 4) and patches (Figure 5) experiments, not merely experiment-dependent. This leads to two issues: There is a computational overhead, which affects runtime and should be reported for runtime comparisons. More importantly, we do not get metric properties for pair-dependent bandwidth. Bandwidth parameters are known to be highly important for performance, so I would expect to see experimental results for fixed bandwidth. I think that the experiment should report results with fixed bandwidth and separate ones for tuned bandwidths.

Generally, the benchmarking is very focused on comparing against DSW, but alternative "Busemann-slicing" methods (Bonet et al, '26) and "Wasserstein-over-Sliced-Wasserstein" (as used eg in Piening, Duong, Steidl, '26) are not included.

Questions for authors

Questions/Remarks

Q1 The "exact" OTDD implementation uses an entropic solver. Given the small entropic regularization used, the benchmarking is still fine in that respect, but could you clarify that in the SuppMat?

Q2 The Pearson correlations use squared distances for VSW and DSW, but unsquared distances for s-OTDD, apparently inherited from the github repo of Piening and Beinert (2026). Could you instead report correlations using consistent powers to ensure a fair comparison? Spearman correlations should not be affected by this, so it is only a minor correction for consistency.

Q3 Is the bandwidth tuning overhead included in runtime analysis?

Q4 a) Why does Figure 7 seem as if DSW has not been run to convergence? In constrast, it seems as MMDSW has been run to convergence even though it is more expensive. Did I miss some convergence criterion?

b) Could you add some ablation of parameter for the minimization in Figure 7? It is a nice and interesting new experiment, but neither MMDSW nor DSW has been tried here so i would expect some kind of parameter tuning here for fairness. In particualr, the baseline DSW implementation seems to rely on a fixed coarse integration grid of [0, 1] for computation. I would expect that any gradient descent training should use either an exact integration or a very fine grid. The code is missing for this experiment too and would have been nice here.

Q5 Could you please add references for the OT-NNA, TLB, STLB, AE methods? They are currently not referenced, which is not good scientific practice. In particular, note that the methods are developed from Memoli's Gromov-Wasserstein which is also not referenced.

Q6 Could you also include the Busemann-based slicing method of Bonet et al. (2026) for some experiments?

Q7 An interesting direction would be to investigate whether VSW can also be used to construct explicit transport plans between measures. For example, Generalized Wasserstein Flow Matching (https://arxiv.org/abs/2605.08424) considers couplings between probability measures over probability measures ("meta-plans"), using Wasserstein-over-Sliced-Wasserstein for point-cloud generation. At the level of individual measures, Expected Sliced Transport Plans (EST) (https://arxiv.org/abs/2410.12176) constructs transport plans by averaging lifted one-dimensional couplings. Could an analogous EST-type construction be developed from VSW, potentially yielding computationally efficient meta-plans for point-cloud generation? (Potentially out of scope, but I would be curious)

Q8 HIPM (Catalano and Lavenant, 2024) is discussed but not compared experimentally. Given its favorable statistical guarantees and related test-function formulation, could you add an experimental comparison? Their implementation is mainly targeted at 1D and would presumably not scale to high-dimensional setting, but it might still be nice to reconsider one of their experiments or provide some novel illustrative toy experiment.

Q9 Euclidean MMD can be sliced too. It might be good to have a conceptual comparison in the SuppMat.

Q10

We could also define a Wasserstein-over-MMD (WoMMD) distance, which might seem closer in spirit to the proposed construction. So, we would have various options of estimating distances: MMDSW, DSW, WoSlicedWasserstein, sOTDD (without labels), VSW, WoW, WoMMD. Presumably all of them have very different geometries, but I would expect similar topologies.

Both MMD and Wasserstein metricizes weak convergence, but the geometric difference between MMD and Wasserstein is nicely illustrated via barycenter problems: Wasserstein induces a Euclidean interpolation with varying support and Euclidean MMD induces mixtures with fixed support. For free support barycenter approximations, we can just do gradient descent on particle locations. For such simulations, we often see that sliced Wasserstein behaves more similarly to Wasserstein than MMD does.

Long story short: Could you run a gradient descent based free support approximation of the barycenter between two translated metameasures for some of the different distances (ideally with some shared convergence criterion, sufficient particles, suitable visualization of the hierarchical measure via color-coding)?

This would add some intuition on the different geometries.

I think that would be a valuable addition to the SuppMat and trace back to the origins of sliced Wasserstein, which was primarily introduced for barycenter-type problems.

7 comments
AuthorsOpenReview ↗

Response to Reviewer kDCN (Part 1)

We thank the reviewer for their positive assessment, and their valuable comments. We address the raised questions and concerns below, and we have revised the paper accordingly.

W1: Figure 7 visualization and evaluation

We agree that both the visualization and evaluation should better reflect the hierarchical structure of the data. In the original submission, we reported point-level W22W_2^2 as a simple measure of the overall generated sample quality, which does not account for the distinction between inner distributions. We have therefore updated Figure 7 to color-code the generated point clouds according to their nearest target component. We have also replaced the point-level W22W_2^2 with Wasserstein-over-Wasserstein (WoW22\mathrm{WoW}_2^2), which explicitly accounts for both the inner and outer distributions. To ensure a consistent evaluation, we compute WoW against the target meta-measure with exactly uniform weights across its eight components (each assigned weight 1/81/8), rather than using a randomly sampled collection of target clouds. This avoids fluctuations in the number of target clouds per component that could otherwise affect the evaluation. We have clarified this protocol in Section 4 and Appendix H.

AuthorsOpenReview ↗

Response to Reviewer kDCN (Part 2)

W2: Pair-dependent bandwidth

Bandwidth computation

We thank the reviewer for raising this concern. We would like to first note that the bandwidth ℓ\ell is set by the median heuristic, which is a single statistic of the input points, with no search and no access to any ground truth required. We first clarify what the original runtimes measure. For every method, the runtimes in Figures 3 and 5 and Tables 1 and 2 time the distance computation only, and we exclude the bandwidth. This is done to isolate the cost of each distance method. In the original submission, ℓ\ell was set to the median of all pairwise distances among 2000 points sampled from the pooled inner measures, i.e. about 2⋅1062\cdot10^6 distances. In Figure 3 this took 33–83 ms per pair of meta-measures. However, we show that the same median can be estimated from 2000 independent random pairs of points, which costs 2–5 ms (10–30× less), much cheaper than the original. We next verify that the two estimators result in the same bandwidths and the same results. We compare them on the same pairs of meta-measures and the same random slices, so that only ℓ\ell changes. In Figure 2, the new ℓ\ell is within 2.1% of the original one for every pair, and the correlations with OTDD are essentially unchanged (Table A).

Table A. Correlation of VSW with OTDD in Figure 2, with the original vs. new bandwidth estimator.

ExperimentDownstream metricOriginal → new
Fig. 2, correlation with OTDDPearson / Spearman, MNIST, L=1000L=10000.904 / 0.924 → 0.900 / 0.928 (largest change over all settings: 0.024)

All experiments in the revision use the new estimator. That is, Figure 2,4,5, and Table 2 numbers for VSW are updated. However, none of the main conclusions or trends are changed. We have also added the new estimator's description to Appendix H.

Runtimes including the bandwidth. We then show that the computation overhead of the new bandwidth is negligible, and we report runtimes with and without the bandwidth computation overhead. Table B reports Figure 3 on MNIST at four dataset sizes, for the smallest and largest slice budgets.

Table B. Figure 3, MNIST: VSW runtime per distance on CPU (ms, mean of 10 draws), by dataset size.

VSW runtime (ms)5,00020,00040,00060,000
1k slices, distance only6.016.137.159.6
1k slices, + bandwidth8.318.539.161.7
10k slices, distance only23.733.052.876.4
10k slices, + bandwidth26.035.454.878.5

The bandwidth step does not depend on the number of slices, and its relative cost shrinks as the datasets grow (+3–4% at 60k samples). Across all datasets, sizes and slice budgets of Figure 3, VSW including the new bandwidth step remains its original trend, faster than DSW and other baselines.

Table C. Table 1 and Figure 5: runtime per distance of VSW without and with the new bandwidth step, and of DSW.

ExperimentVSW, distance onlyVSW + bandwidthDSW
Table 1, ShapeNet chairs (s)0.01250.01340.0709
Fig. 5, image patches (ms)6789298

Table D. Table 2: Runtime per distance (ms). The bandwidth is computed once per dataset, so we spread its cost over all pairs of shapes.

Method2D ShapesAnimalsFAUST-500FAUST-1000MNIST-2000
VSW, distance only0.350.342.803.767.48
VSW, with new bandwidth0.360.352.813.777.49
SQW0.930.956.5110.1518.46

With the new estimator, the bandwidth adds 2–5 ms per pair in Figure 3, 0.9 ms in Table 1, 22 ms in Figure 5, and at most 0.01 ms per distance in Table 2. With the new bandwidth computation overhead included, none of the comparisons in the paper changes.

Fixed bandwidth experiments

We have now added an ablation with a fixed bandwidth to Appendix F.2, Figures 8–9. We use the point-cloud and patch experiments. For each sweep, we compute a single bandwidth ℓ0\ell_0 by the median heuristic on the reference meta-measures only (the side of each comparison that is kept fixed along the sweep), and use it for every pair. Since ℓ0\ell_0 never depends on the targets, VSW is a single metric along the whole sweep. The pairs and the random slices are the same for every bandwidth, so only the bandwidth changes. The fixed ℓ0\ell_0 generally gives the same trends as the per-pair bandwidth in all sweeps.

Bandwidth sweep. We also report every bandwidth ℓ=c ℓ0\ell = c\,\ell_0 for c∈{1/8,1/4,…,8}c \in \{1/8, 1/4, \dots, 8\} separately. VSW recovers the trends of the main text for bandwidths within a factor of two of ℓ0\ell_0 and degrades gradually outside this range. VSW with a much smaller bandwidth detects that the target has changed, but not by how much. A much larger one only sees coarse differences. Tuning within this range changes the results only slightly, so the median heuristic, which needs no tuning, is already a good choice.

AuthorsOpenReview ↗

Response to Reviewer kDCN (Part 3)

W3: Additional baselines

We thank the reviewer for their suggestion. We have now added Busemann-slicing baselines (Bonet et al, 2026) to the dataset distance experiment (Figure 2, 3). Additionally, we added Wasserstein-over-Sliced-Wasserstein (WoSW) to the point-cloud and patch experiments (Figure 4,5 and Table 1), and summarized results are provided below. WoSW is also added to the barycenter computation experiments (Appendix G.2) (For additional details, please refer to our response to Q10).

Busemann-slicing methods. We include SWB1DG, which uses one-dimensional Gaussian geodesic rays, and SWBG, which uses a Gaussian approximation of each class after a t-SNE embedding. We use the official public implementation without modification and with its default hyperparameters. As for every other method in Figure 2, samples are projected linearly from the flattened images.

Both Busemann distances are more accurate than s-OTDD, and generally more accurate than DSW at small and moderate budgets. For few slices, VSW remains the most accurate method or is on par with the best one, and for large budgets, VSW is competitive with the Busemann distances.

We also timed both methods under the same protocol as Figure 3. At every dataset size and budget, SWB1DG is slower than s-OTDD and orders of magnitude slower than VSW. SWBG is dominated by the t-SNE embedding, and becomes too slow beyond small datasets. We therefore stop calls that take longer than 10×10\times the slowest s-OTDD call, and mark such calls as timed out in the figure.

Overall, the Busemann distances can match VSW's accuracy only with many more slices and at a much higher cost per evaluation. VSW therefore still offers the best accuracy–runtime trade-off. The revised paper includes both methods in Figures 2 and 3, and the implementation details in the appendix.

WoSW baseline. We evaluate WoSW on exactly the same data as the other methods in Figure 4, Figure 5 and Table 1, with L=100L=100 projections. VSW still remains the fastest method: 0.0125 s vs. 0.145 s for WoSW on the ShapeNet benchmark (Table 1, 11.6×), and 67 ms vs. 159 ms on the image patches (Figure 5, 2.4×), where VSW also follows exact WoW more closely. On the low-dimensional point clouds of Figure 4, all methods show the same trends.

Questions

Q1: "exact" OTDD clarification

We thank the reviewer for their suggestion. We have revised Appendix H (“Optimal Transport Dataset Distance”) to clarify this.

Q2: Consistent powers in Pearson correlations

We thank the reviewer for bringing this to our attention. We have now updated Figure 2 to use the squared s-OTDD distance.

Q3: Is the bandwidth tuning overhead included in runtime analysis?

Please refer to our response to W2.

Q4(a) and (b): Figure 7: DSW convergence and parameter ablation

Q4(a): We appreciate the opportunity to clarify this. For this experiment, all methods are trained for the same fixed budget of 100,000 steps, and there is no early stopping. The curves end at different points since the x-axis is wall-clock time. Since MMDSW is the slowest per step, its curve extends furthest. In the revision, each method is still tuned within this 100k-step budget (see (b)). To make sure that DSW's result is not a convergence issue, we also trained DSW with its selected settings for 300,000 steps (3 seeds). Its final WoW22\mathrm{WoW}_2^2 does not change, and remains above both VSW and MMDSW. The gap is therefore not due to training length.

Q4(b): We agree with the reviewer’s comment, and we sweep the main hyperparameters of each method on both datasets:

  • VSW: slices L∈{32,128,256,512}L\in\{32, 128, 256, 512\}, random features R∈{128,512}R\in\{128, 512\}, the length of the initial large-bandwidth phase (10, 20 or 30% of the steps), the final bandwidth, and the learning rate (3⋅10−43\cdot10^{-4}, 10−310^{-3}, 3⋅10−33\cdot10^{-3}).
  • DSW: GP length scale ∈{0.003,0.01,0.03,0.1}\in\{0.003, 0.01, 0.03, 0.1\}, quantile grid of 50 or 200 points, slices (outer × inner) ∈{100×100,200×200,1000×100}\in\{100{\times}100, 200{\times}200, 1000{\times}100\}, and the learning rate (10−310^{-3}, 3⋅10−33\cdot10^{-3}).
  • MMDSW: Riesz exponent r∈{0.5,1,1.5}r\in\{0.5, 1, 1.5\}, projections ∈{32,128,512}\in\{32, 128, 512\}, and the learning rate (10−310^{-3}, 3⋅10−33\cdot10^{-3}).

For every method, we select the configuration with the lowest final WoW22\mathrm{WoW}_2^2 and rerun it with 3 seeds for Figure 7. Table 4 in Appendix H lists the swept and selected values. Overall, every method is tuned with the same step budget, selection criterion and number of seeds to ensure the comparison is fair. After tuning, VSW matches MMDSW and improves on DSW, while training 2–5× faster than MMDSW in wall-clock time. We will also publicly release the code for all experiments, including the Figure 7 experiments and hyperparameter sweeps, upon acceptance.

Reviewer_Ne5kOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This submission proposes a new type of slicing procedure for comparing distributions on the space of distributions adding to the recent literature on space of distributions on distributions. The primary idea is to assign to each P∈P(P(Ω))\mathbb P\in\mathcal P(\mathcal P(\Omega)) (a distribution over P(Ω)\mathcal P(\Omega)) a one-dimensional measure for each slice and then to compare these one-dimensional measures using Wasserstein distances which have a closed form solution and aggregate over slices.

Explicitly, fix a compact metric space (Ω,ρ)(\Omega,\rho) and let κ:Ω×Ω→R\kappa:\Omega\times \Omega\to\mathbb R be a continuous, symmetric, and universal positive definite kernel with associated centered Gaussian process on C(Ω)C(\Omega), σ=GP(0,κ)\sigma= \mathrm{GP}(0,\kappa), which admits almost surely continuous paths. Then, the (p-th power of the) vertical sliced Wasserstein distance is defined by VSWpp:(P,Q)∈P(P(Ω))×P(P(Ω))↦∫C(Ω)Wpp((Pf)♯P,(Pf)♯Q)dσ(f),\mathrm{VSW}_p^p:(\mathbb P,\mathbb Q)\in\mathcal P(\mathcal P(\Omega))\times \mathcal P(\mathcal P(\Omega))\mapsto \int_{C(\Omega)} \mathrm{W}_p^p( (P_f)_{\sharp} \mathbb P,(P_f)_{\sharp} \mathbb Q)d\sigma(f), where (Pf)♯P∈P(R)(P_f)_{\sharp} \mathbb P\in\mathcal P(\mathbb R) which is defined by (Pf)♯P(B)=P(μ∈P(Ω):∫Ωfdμ∈B)(P_f)_{\sharp} \mathbb P(B)=\mathbb P(\\{\mu\in\mathcal P(\Omega):\int_{\Omega}fd\mu\in B\\}) for each Borel subset BB of R\mathbb R.

With this definition in hand, it is shown that VSWp\mathrm{VSW}_p defines a metric on P(P(Ω))\mathcal P(\mathcal P(\Omega)) and metrizes the weak topology on P(P(Ω))\mathcal P(\mathcal P(\Omega)). It is shown, moreover that (P(P(Ω)),VSW2)(\mathcal P(\mathcal P(\Omega)),\mathrm{VSW}_2) embeds isometrically into a separable Hilbert space and that VSW2β\mathrm{VSW}_2^{\beta} is conditionally negative definite for each β∈(0,2]\beta \in(0,2] so that exp⁡(−γVSW2β)\exp(-\gamma \mathrm{VSW}_2^{\beta}) is positive definite for all γ>0\gamma>0.

Next, an algorithm is proposed to compute VSWp\mathrm{VSW}_p when P\mathbb P and Q\mathbb Q are each supported on a finite number of measures NN and Ω⊂Rd\Omega\subset \mathbb R^d. It consists of sampling a slicing function using RR-feature slices, then one computes the pp-Wasserstein distance between the resulting sliced measures which are univariate discrete measures (via sorting), and finally one repeats the process LL times and averages the contributions to approximate the integral over σ\sigma. The complexity of the proposed algorithm is then O(NnRd+LNR+LNlog⁡N)O(NnRd+LNR+LN\log N).

Additionally, it is shown that in the two-stage setting, where μ1,…,μN\mu_1,\dots, \mu_N are i.i.d. from P\mathbb P and xi1,…,xinx_{i1},\dots, x_{in} are i.i.d. from μi\mu_i and conditionally independent given μi\mu_i and we approximate P\mathbb P by P~N,n=1N∑i=1Nδμ^i,n\tilde{\mathbb P}_{N,n}=\frac 1N\sum_{i=1}^N\delta_{\hat \mu_{i,n}} where μ^i,n=1n∑j=1nδxij\hat \mu_{i,n}=\frac{1}n\sum_{j=1}^n \delta_{x_{ij}}, it holds that E[VSWpp(P,P~N,n)]=O(N−1/2+n−p/2),\mathbb E[\mathrm{VSW}_p^p(\mathbb P,\tilde{\mathbb P}_{N,n})]= O(N^{-1/2}+n^{-p/2}), where the constants depend on pp and σ\sigma. Moreover, if f1,…,fLf_1,\dots, f_L are i.i.d. from σ\sigma, E[∣1L∑l=1LWpp((Pfl)♯P,(Pfl)♯Q)−VSWpp(P,Q)∣]=O(1/L),\mathbb E[|\frac 1L\sum_{l=1}^L \mathrm{W}_p^p((P_{f_l})_{\sharp} \mathbb P,(P_{f_l})_{\sharp} \mathbb Q )-\mathrm{VSW}_p^p(\mathbb P,\mathbb Q)|]=O(1/\sqrt L), where the constants depend again on pp and σ\sigma.

Finally, the gradient of VSW22\mathrm{VSW}_2^2 under perturbations of the support points of one of the measures is computed with the premise of performing particle gradient flow to minimize this metric.

The paper concludes with a number of examples which establish that the vertical sliced Wasserstein distance exhibits similar trends to other methods for comparing distributions over distributions with the advantage of faster computation on a large number of tasks.

Critical strengths

This formulation of a distance on the space of distributions over distributions is novel to my knowledge and appears sensible. It is particularly interesting that the slicing procedure reduces the distributions over distributions into univariate distributions which can be easily compared using Wasserstein distances. The resulting algorithm is conceptually simple and appears preferable to other methods (at least in terms of runtime as shown in the experiments).

Critical weaknesses

In my opinion, the level of discussion in this paper is minimal and there are a number of unclear passages. The main examples are

  1. After Assumption 1, there are no examples provided in the main text for kernels satisfying the assumption. This leaves readers wondering how strong this assumption is. In particular, one might ask if, for any desired Ω\Omega, such a kernel exists or if the choice of kernel has some impact on the performance of this distance in experiments.
  2. Why is Wasserstein over Wasserstein introduced on p. 3 after introducing the assumption on the kernel? There is no lead-in to this definition, surely it could be placed somewhere more natural.
  3. Theorem 3.6 states that, (P(P(Ω)),VSW2)(\mathcal P(\mathcal P(\Omega)),\mathrm{VSW}_2) embeds isometrically into a separable Hilbert space and that VSW2β\mathrm{VSW}_2^{\beta} is conditionally negative definite for each β∈(0,2]\beta \in(0,2] so that exp⁡(−γVSW2β)\exp(-\gamma \mathrm{VSW}_2^{\beta}) is positive definite for all γ>0\gamma>0. What are the implications of this result at the practical or theoretical level?
  4. The section on computation starts with the sentence "For translation-invariant κ\kappa on Ω⊂Rd\Omega\subset \mathbb R^d with spectral measure τk\tau_k (Bochner) an RR-feature slice is [...]". The reader has no context for what the point of this RR-feature slice definition is. Moreover, the (Bochner) is not a citation to anything in the references. I recommend to rewrite this section more carefully, given that this is a crucial step for implementing the algorithm.
  5. In Section 3.3, why are two plug-in meta-measures defined, from what I can see P^N\hat{\mathbb P}_N is not used in the main text.
  6. In Remark 3.8. the acronym HIPM is never defined.
  7. What is the purpose of the sentence "This is an empirical scaling study: Theorem 3.7 assumes a compact domain, and Proposition 3.9 bounds the powered-distance estimator."? Why not just conduct the experiment under the actual setting of the paper instead?
  8. The start of Section 3.4 should describe what the setup of the problem is before explaining how the problem is solved.
  9. In Proposition 3.10, what are Assumptions 1-2? There is only one assumption environment in the paper and no Assumption 2 in the main text.

Another important weakness of this work is that no guarantees are provided for the output of Algorithm 1. To my understanding step 2 in Algorithm 1 does not generate i.i.d. samples from σ\sigma so that Proposition 3.9 cannot be applied.

Questions for authors
  1. While the two-stage sampling rate can be combined with the Monte Carlo error to give an overall approximation error, it seems that this does not apply to the output of Algorithm 1 given that the flf_l are only approximations of samples? Also, can the notation between the output of the algorithm VSW^pp\widehat{\mathrm{VSW}}_p^p and the Monte Carlo approximate VSW^Lp\widehat{\mathrm{VSW}}_L^p be updated so that they no longer clash?

  2. It would be helpful for readers to have a brief introduction to optimal transport in an appendix. While equation (2) can, technically, serve as a definition for the pp-Wasserstein distance between univariate measures, I believe readers unfamiliar with this topic would appreciate seeing the general formulation and, e.g., a citation to a reference where the univariate formula is derived.

  3. I recommend that the authors revise the paper taking into account the points brought up in the weaknesses section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Ne5k (Part 1)

We thank the reviewer for their time and valuable comments. We have revised the paper according to the comments, and address the questions and concerns below.

W1: No example kernels after Assumption 1

We agree that providing a concrete example helps clarify Assumption 1. We have therefore moved the example of a Gaussian RBF kernel to the main text, following Assumption 1 in Section 2.

W2: Placement of the WoW definition

We thank the reviewer for pointing this out. We have moved the definition of Wasserstein-over-Wasserstein (WoW) to immediately follow the discussion of one-dimensional optimal transport in Section 2.

W3: Implications of Theorem 3.6

The Hilbertian structure of VSW2\mathrm{VSW}_2 has several theoretical and practical implications. In particular, it allows us to construct positive-definite kernels on meta-measures of the form Kγ,β(P,Q)=exp⁡(−γ,VSW2β(P,Q))K_{\gamma,\beta}(\mathbb{P},\mathbb{Q})=\exp(-\gamma \\, \mathrm{VSW}_2^\beta(\mathbb{P},\mathbb{Q})). We further show that these kernels are universal and characteristic, enabling kernel-based methods such as two-sample testing and distribution regression on meta-measures. Moreover, the induced MMD defines a metric on distributions over meta-measures (i.e., distributions of distributions of distributions), extending the framework to an additional hierarchical level. This provides a potential direction for future work in applications involving such higher-order distributions. We provide the formal result and further discussion in Appendix C (Corollary C.10 and Remark C.11).

W4: RR-feature slice / Bochner in the computation section

We have revised Section 3.2 to first explain the use of random Fourier features (RFFs) to approximate Gaussian-process test functions and define RR as the number of features per test function. We then introduce Eq. (5), with an explicit reference to Bochner's theorem and the corresponding citation.

W5: In Section 3.3, why are two plug-in meta-measures defined, from what I can see P^N\hat{\mathbb{P}}_N is not used in the main text.

We thank the reviewer for pointing this out. We agree that P^N\widehat{\mathbb{P}}_N is not needed in the main text, and it is an auxiliary empirical meta-measure used in the proof of Theorem 3.7. We have therefore moved its definition from Section 3.3 to the beginning of the proof of Theorem 3.7 in Appendix D, where it is also used in Remark D.2. Section 3.3 now introduces only P~N,n\widetilde{\mathbb{P}}_{N,n}, which is directly used in the theorem.

W6: In Remark 3.8. the acronym HIPM is never defined.

We thank the reviewer for bringing this to our attention. We have now introduced the HIPM acronym in the Introduction, which stands for Hierarchical Integral Probability Metric (Catalano and Lavenant, 2024). We have also added the corresponding citation in Remark 3.8.

W7: What is the purpose of the sentence "This is an empirical scaling study: Theorem 3.7 assumes a compact domain, and Proposition 3.9 bounds the powered-distance estimator."? Why not just conduct the experiment under the actual setting of the paper instead?

We agree that the original sentence was unclear. The purpose of Figure 1 is to examine two distinct sources of estimation error: Theorem 3.7 concerns two-stage sampling from the outer and inner measures (varying NN and nn), whereas Proposition 3.9 concerns Monte Carlo approximation using LL slices. We have revised the text in Section 3.3 to make this distinction explicit. The experiments use the same two-stage sampling scheme, but Gaussian inner measures have unbounded support and the slices are approximated with finite Fourier features. Accordingly, we present the results as an empirical illustration of the scaling behavior, rather than as a direct verification under all theoretical assumptions.

AuthorsOpenReview ↗

Response to Reviewer Ne5k (Part 2)

W8: The start of Section 3.4 should describe what the setup of the problem is before explaining how the problem is solved.

We have revised the beginning of Section 3.4 to explain that the goal is to transport a source meta-measure toward a fixed target using VSW as the objective. We also clarify that, in the empirical setting, each meta-measure consists of a collection of distributions represented by finite sets of particles (e.g., points or images), providing context for the subsequent particle-level gradient flow formulation.

W9: In Proposition 3.10, what are Assumptions 1-2? There is only one assumption environment in the paper and no Assumption 2 in the main text.

We have moved Assumption 2 to Section 2 of the main text, ensuring that the necessary assumptions are stated before they are used.

W10: Another important weakness of this work is that no guarantees are provided for the output of Algorithm 1. To my understanding step 2 in Algorithm 1 does not generate i.i.d. samples from σ\sigma so that Proposition 3.9 cannot be applied.

We agree that the original statement and the step 2 of the Algorithm were not precise. We have revised Section 3.2 and Algorithm 1 to clarify that our implementation uses the random Fourier feature (RFF) approximation in Eq. (5), and have removed the alternative truncated Karhunen–Loeve description to avoid ambiguity. With independently sampled Gaussian coefficients, the resulting slices are i.i.d. conditional on the shared Fourier basis. Consequently, the Monte Carlo variance argument of Proposition 3.9 applies conditionally to the finite-feature slicing distribution, yielding the same L−1/2L^{-1/2} rate for its powered-distance estimator. We acknowledge that this does not directly bound the approximation error relative to the ideal GP-based VSW, which remains a separate source of error. We clarify this distinction in Appendix D.

Questions

Q1: Overall approximation error of Algorithm 1, and notation clash

We agree that the combined sampling and Monte Carlo error bound applies to the ideal estimator with i.i.d. GP slices, and does not directly account for the finite-feature approximation used in Algorithm 1. As clarified in Appendix D, the slices generated by Eq. (5) are i.i.d. conditional on the shared Fourier basis, so the variance argument of Proposition 3.9 applies to the corresponding finite-feature estimator. However, this does not bound the additional approximation error introduced by finite RR, which we acknowledge as a limitation. We have also revised the notation to distinguish the ideal Monte Carlo estimator VSW^p,Lp\widehat{\mathrm{VSW}}_{p,L}^{p} from the finite-feature estimator VSW^p,L,Rp\widehat{\mathrm{VSW}}_{p,L,R}^{p} computed by Algorithm 1, and clarified the scope of the combined error bound in Appendix E.

Q2: Introduction to optimal transport in the Appendix

We thank the reviewer for this helpful suggestion. We agree that a brief introduction to optimal transport would make the paper more accessible to readers unfamiliar with the topic. We have therefore added a new section, "Introduction to Optimal Transport," at the beginning of the appendix. This section introduces the general Kantorovich formulation of the Wasserstein distance, explains the one-dimensional formulation used in Eq. (2), and provides relevant references for further details and derivations.

Q3: I recommend that the authors revise the paper taking into account the points brought up in the weaknesses section.

We thank the reviewer for their valuable suggestions, which have helped us revise and improve the paper. The revised version is submitted, and we are happy to provide further clarifications and revisions as needed.

References

Catalano, M. and Lavenant, H. (2024) ‘Hierarchical integral probability metrics: A distance on random probability measures with low sample complexity’, ICML.

Reviewer_Ne5kOpenReview ↗

Comments on response

I thank the authors for clarifying all of these points, I believe this will help to improve the quality of the submission. I have no further questions at this time and will revise my initial assessment of this work to take these new points into account.

We sincerely thank the reviewer for their time and consideration. We appreciate their constructive feedback, which has helped us improve the quality and clarity of the manuscript.

Reviewer_eNhcOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper deals with an approximation of the wasserstein distance of meta-measure (measure of measure), whose compututional cost is very expensive to evaluate. The authors proposes to approximate each of the inner measure by a scalar via projections using continous test functions, enabling the outer wasserstein distance computation to benefit from the properties of 1-D Wasserstein distance. The proposed approximation is called Vertical Sliced Wasserstein, and studied in the article.

The article is structured as follow,

  • Introduction and related work
  • Background and Standing Assumption
  • Vertical Sliced Wasserstein (well-posedness , metricity, topology ; computation ; statistical properties ; gardient flows)
  • Experiments
  • Conclusion and Future Works

with Appendices convering Proofs and adding details to the related works among other things (relationships with MMD, ablation study of the R parameter).

Main Part

VSW is showned to be well-posed (Lemma 3.2), a metric (Theorem 3.3) and embbed isometrically into a separable Hilbert space (Theorem 3.6).

The main advantage of the proposed formulation is the absence of assumptions on the meta-measures. (There is assumption of the universality of the kernel behind the vertical slicing (test functions are chosen to follow the law of a Gaussian Process with a universal covariance kernel)).

On the computational side, the algorithm can be seen as an empirical mean of the 1-D wasserstein ditance of associated to each of the L vertical slices (corresponding to a test function). The test functions are parametrized with R and are sampled for each slice. The covariance of the associated law of the test functions converges to the universal kernel when R tends to infinity. The scalar associated to each inner distribution is the empirical mean of the image of each support by the test function. In order to obtain some practical gain, the Fourier Basis behind the test functions is shared.

The quality of the proposed estimator is studied theorically and expreimentally (varying the number of slices, the number of outer clouds (number of distributions), the size of the inner distributions). The only parameter not studied is R (which deals with the quality of the vertical slicing, the study is in the Appendices).

Elements of the Gardient Flows setting are then given with Algorithmic details in App. C (Alg. 2), based on the form of the gardient given previously.

Experiments

The experiments follows mainly the DSW experiments complemented with particles / gradient flows experiments. Among them, an example is the correlation with the so-called Optimal Tansport Dataset Distance, which try to infer an priori distance between dataset in advance of a domain adaptation step or fine-tuning.

Other experiments deal with comparison of point clouds, with image distribution seen as meta-measures of patch distributions.

Comparison is provided with DSW and WoW. Trends appear to be similar, with computation time in favor of the proposed VSW.

The conclusion mainly offers a quick summary .

Critical strengths

The slicing of WoW has already been adressed in several works.

The main novelty here is that there are no ‘slicing per se’ (what the authors call horizontal slicing), only the application of tests functions which allows for a simplified computationnal framework (1 D Wasserstein). To mitigate the dependance on the test function, this is done in expectation, leading to a proper characterisation of VSW. Somehow, it removes the need for comparison and computation between each of the meta-measure, which is one element of the expensive computation of WoW. The theoritical study of VSW is well done and highlights the dependancies at both level, with added experimental verification. As shown in the experiments, its reduced complexity allows it to compare meta-measure efficiently.

The main critical strength is in the real "simplicity" of the idea behind the proposal and further simplification of previous proposition like DSW. This simplicity is paired with a fast Monte Carlo approximation rate that scales with regard to the number of both supports (inner (n) and outer (N)). Furthermore, the various properties of VSW (and relationships to MMD given in Appendices) offer a sound study of the VSW. To explicit the simplification gap: the main comparison is with DSW which represents inner measure through quantile functions and then applies functional slicing with Gaussian-process direction. The capacity to apply directly functional slicing in the proposed method with theoritical grounding is a strength.

Another advantage with respect to other slicing approximation like DSW resides in the explicit gradient for VSW2 and gradient flows applications.

Critical weaknesses

The main weaknesses :

  • a first limit is that there are no theoritical bounds with regards to WoW, which is what VSW tries to approximate at first. The test-functions operate in a way that circumvent any intermediary procedure. For example, DSW proposes intermediary distances that somehow act as intermediary approximation.

  • the relationships with other slicing methods of WoW could be exposed in more details.

  • a second limit is that the functionnal part is depending (experimentally) on the quality of the test-functions linked to parameter R. While the Ablation study in the Appendice provides a lot of elements, there are few insights in the main article in particular with respect with the dimension of the metric space of the inner distributions.

Questions for authors

Clarifications .

  • Can others types of tests functions be used ?
  • Can bounds related to other sliced methods be established easily ?

Suggestions for future works

One limitation of meta-measures and OTDD is in the common metric space of inner distributions. Could the test-function approach be used with theoritical guarantee to enable a more general setting ?

4 comments
AuthorsOpenReview ↗

Response to Reviewer eNhc (Part 1)

We thank the reviewer for their positive assessment, and their valuable comments. We address the raised questions and concerns below, and we have revised the paper accordingly.

W1. Theoretical bounds with respect to WoW

We have expanded Appendix C with theoretical comparisons relating VSW to WoW. Proposition C.4 shows VSW2≤WoW2,MMDκ≤Lκ,WoW2\mathrm{VSW}_2 \le \mathrm{WoW}_{2,\mathrm{MMD}_\kappa} \le L_\kappa \\, \mathrm{WoW}_2, where WoW2,MMDκ\mathrm{WoW}_{2,\mathrm{MMD}_\kappa} serves as the intermediate distance. Conversely, Proposition C.8 shows that on compact Ω\Omega, WoWp≤ω(VSWp)\mathrm{WoW}_p \le \omega(\mathrm{VSW}_p) for a modulus ω\omega with ω(0+)=0\omega(0^+)=0. Remark C.9 shows that this reverse bound cannot be improved to a Hölder rate in general.

W2: the relationships with other slicing methods of WoW could be exposed in more details.

The other existing slicing constructions differ in where they simplify the transport problem: WoSW (Wasserstein over Sliced Wasserstein) replaces the inner Wasserstein ground distance by sliced Wasserstein (SW) while retaining outer OT. DSW uses projected inner quantiles and subsequent functional slicing. Busemann methods use scalar functions associated with Wasserstein geodesic rays. We made these distinctions explicit in the new Appendix I.

W3: Dependence on test-function quality (parameter RR)

We thank the reviewer for highlighting the importance of the finite-feature approximation and its dependence on the choice of RR. We have revised Section 3.2 to clarify that random Fourier features (RFFs) are used to approximate the Gaussian-process test functions, explicitly define RR as the number of Fourier features per test function, and discuss the trade-off between approximation quality and computational cost. Regarding the dependence on the underlying dimension dd, we acknowledge that our current theoretical results do not characterize how the number of Fourier features RR scales with dd. This remains an interesting question that we plan to explore.

AuthorsOpenReview ↗

Response to Reviewer eNhc (Part 2)

Q1: Can others types of tests functions be used ?

Yes. Gaussian processes are a convenient way to get the property we need. They are not part of the definition. VSW can be defined with any Borel probability measure σ\sigma on C(Ω)C(\Omega). The requirement for VSW to be a metric is that σ\sigma has full support, supp(σ)=C(Ω)\mathrm{supp}(\sigma) = C(\Omega). The argument is short. For probability measures μ\mu, we have ∣⟨g,μ⟩−⟨h,μ⟩∣≤∥g−h∥∞|\langle g,\mu\rangle - \langle h,\mu\rangle| \le \|g-h\|_\infty. So the sliced discrepancy is continuous in the test function gg. If VSW vanishes, the slices therefore agree σ\sigma-a.e., and full support extends this to every g∈C(Ω)g \in C(\Omega). That identifies the two meta-measures. Without full support, VSW is in general only a pseudometric. Any finite family of test functions is one such case.

Within this condition, many other choices of σ\sigma are valid:

  • Discrete measures on a dense family. C(Ω)C(\Omega) is separable when Ω\Omega is compact. Take a countable dense family (gj)j≥1(g_j)_{j\ge1} and weights wj>0w_j>0 with ∑jwj=1\sum_j w_j = 1. Then σ=∑jwj,δgj\sigma = \sum_j w_j \\, \delta_{g_j} has full support, since every open ball contains some gjg_j. Polynomial or wavelet dictionaries are natural examples.

  • Random neural networks. Let σ\sigma be the law of a random one-hidden-layer network whose width nn is drawn with πn>0\pi_n>0 for every nn. Its parameters are drawn from a full-support distribution, for example Gaussian. By universal approximation, the union over widths is dense in C(Ω)C(\Omega). The parameter-to-function map is continuous. Together these give σ\sigma full support.

  • Other Gaussian measures. A Gaussian measure on C(Ω)C(\Omega) has full support exactly when its Cameron–Martin space (the RKHS) is dense in C(Ω)C(\Omega). This holds for any universal kernel, so the GP construction itself leaves a lot of freedom.

The choice of σ\sigma also brings in the large literature on efficient slicing for the Sliced Wasserstein distance. Examples include random-path projecting directions (Nguyen et al., ICML 2024) and quasi-Monte Carlo slicing (Nguyen et al., ICLR 2024). Extending these sampling strategies to function-space slicing is a natural direction. We plan to add a remark summarizing this discussion to the appendix of the final manuscript and leave further investigation of this direction to future work.

Q2: Can bounds related to other sliced methods be established easily ?

Please refer to our response to W2. We have added a few theoretical comparisons with related sliced methods in Appendix I, and a direct comparison with Wasserstein-over-Sliced-Wasserstein (WoSW).

Q3: One limitation of meta-measures and OTDD is in the common metric space of inner distributions. Could the test-function approach be used with theoritical guarantee to enable a more general setting ?

We thank the reviewer for raising this interesting direction. Our current formulation assumes that all inner measures are defined on a common compact metric space Ω\Omega, since the same test functions are applied across these measures. One possible extension is to embed measures from different underlying spaces into a shared representation space and apply VSW to their pushforward measures. Under suitable injective embeddings and kernel assumptions, our existing metricity guarantees would continue to apply in this shared space. However, extending the framework to heterogeneous spaces without a common representation would require defining compatible test functions and establishing additional theoretical guarantees. We view this as a promising direction for future work.

References

Nguyen, K., Bariletto, N. and Ho, N. (2024) ‘Quasi-Monte Carlo for 3D sliced Wasserstein’, ICLR.

Nguyen, K., Zhang, S., Le, T. and Ho, N. (2024) ‘Sliced Wasserstein with random-path projecting directions’, ICML.

Thanks, for the responses, that clearly answers my questions. I have checked the paper accordingly. As I have already accepted the paper, i will follow the consensus toward acceptance and raise my score from weak accept.

We sincerely thank the reviewer for their support of our work and for raising their score. We are glad that our responses have provided clarifications to their questions.

Comments on the paper

1 comment

We thank all reviewers for their valuable feedback. We have uploaded a revised version of the manuscript addressing the raised comments and suggestions. For better readability, revised or added text, and the titles of newly added sections and subsections are highlighted in blue. Updated figures are identified by blue captions, and revised or new values in tables are also highlighted in blue. The revisions include additional theoretical analysis and clarifications, expanded comparisons with related methods, and new experiments and ablations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.