acceptodds
Under review as a conference paper at ICLR 2027

Sliced Orlicz-Wasserstein

Abstract

We propose sliced Orlicz-Wasserstein (SOW) distance which is a generalization of sliced Wasserstein (SW) distance. SOW replaces the norm in SW with a Luxemburg norm cost induced by an Orlicz function . First, we prove that SOW distance is a metric on the space of measures with finite Orlicz norm, and show that it recovers the SW distance when the Orlicz function is . Next, we derive the topological properties of the SOW distance. In particular, we show that convergence under SOW implies weak convergence, and the converse is true under the compact support condition. We then present the theoretical results for estimating the SOW distance. We derive sample complexity for both the distance itself and the powered functional of the distance, and prove their minimax optimality. In addition, we discuss the computational algorithm for approximating the SOW distance by Monte-Carlo estimation and bisection search, as well as the associated approximation error and computational complexity analysis. Our experimental results reveal the superior computational efficiency of SOW compared with Orlicz-Wasserstein (OW) distance. Also, in the experiments, we demonstrate the favorable flexibility of SOW distance over SW in detecting differences between distributions by comparing their performance in two-sample tests and evaluating generative models on image datasets.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Optimal Transport, Sliced Optimal Transport, Orlicz Norm, Tail Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_g3XiOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies the sliced orlicz Wasserstein distance, an extension of the sliced Wasserstein distance in which the cost function is determined by an orlicz function \phi (i.e. a function from the positive values into the positive values that is strictly increasing, convex and null at zero).

After introducing the sliced version of such metrics, the authors delve into a study of the properties of such metric, establishing an almost one-to-one extension of the classic properties for the sliced wasserstein distance to this specific case.

Owing to the one dimensional nature of the sliced distances, the writing of the orlicz Wasserstein simplifies and takes an easier to handle formula that exploit the standard difference between the pseudo inverse function to simplify the solution of a minimization problem (which is done via a bisection algorithm).

To conclude, the authors provide some experiments to justify the adoption of this metric.

Critical strengths

The paper is well-written and easy to follow. I have found no major presentation issues, making it pleasant to read.

The theoretical results presented are comprehensive and easy to follow/believe: most of the proven properties are straight up generalisation of the standard properties that are known for the sliced Wasserstein distance.

Critical weaknesses

The paper has several issues that I think are not easy to address, however a compelling reply could make me upgrade the score.

The main concern I have is the justification of the paper. The main argument put forward to consider an Orlicz-based Wasserstein over a standard Wasserstein one is to allow for more aggressive costs that penalize the tails of the distributions. To support this claim, the authors point at theoretical applications e.g. "theoretical guarantees for parameter estimation" and "convergence of hypoercive differential equations".

While this can be a theoretical benefit, it comes with an additional computational cost. In my opinion this point is overlooked in the introduction as the authors hide the need to solve the minimization problem that each Orlicz-Wasserstein (OW) distance entails in the landau term of the complexity. In other words, it should be made clear that unless the phi function has a specific form, the sliced Orlicz-Wasserstein (SOW) is always more expansive than a Sliced Wasserstein (SW). Moreover, the cases presented in the paper that are easy to handle are the ones associated to polynomial functions, which are connected to standard Wasserstein distances. This aspect also feels overlooked in the paper.

Indeed, the experiments presented only consider the W_2 distance, never taking into account other exponents (e.g. p=4,6,8,...), the W_\infty nor costs induced by more generic convex functions to make the comparison fair.

Moreover, in the experimental section, the figures show that the computational cost of the SOW is much larger than the standard SW when the Orlicz function is not polynomial. As already said, the case in which phi is not polynomial would be the main case of interest, as the polynomial case can be reconduced to the standard Wasserstein distance, thus this increased computational cost cannot be considered as collateral damage.


On the more theoretical side, while I appreciate the detailed analysis proposed on the SOW, most of the presented results read as an extension of the classic properties known for the SW. While these properties are nice to have and I do understand that going from SW to SOW is not mathematically trivial (especially the statistical properties which I assume are based on a generalisation of the techniques proposed by Bobkov and Ledoux), I do not think that these technicalities are deep enough to justify the paper without a strong experimental section.

Questions for authors
  • Aside penalizing more aggresively the tails of the distributions, is there any explicit benefit that may outweight the need to solve a minimization problem for each slice? The theoretical properties presented are closely related to the ones that the standard SW possesses, what makes the proposal new?

  • why the experimental results consider only the W_2 distance and not a W_6 distance, the W_\infty distance or a distance induced by h(|x-y|) with h convex, increasing and null at zero? These metrics have a closed formula solution for one dimensional problems, see Santambrogio, F. (2015). Optimal Transport for Applied Mathematicians: Calculus of Variations

  • how is the SOW or OW induced by (x^2+x^4)/2 connected to W_2, W_4 or to the Wasserstein distance induced by the cost (x-y)^2+(x-y)^4?

  • why in the second picture of figure 1 there is no SW in the comparison?

minor comments

in the first picture of figure 1 the legend is covering the graphs of the OWs

6 comments
AuthorsOpenReview ↗

Responses to Reviewer's Main Concerns

We would like to thank the reviewer for the time and the willingness to discuss our work. Based on your constructive feedback, we provide our detailed responses below.

W1 The main concern I have is the justification of the paper. The main argument put forward to consider an Orlicz-based Wasserstein over a standard Wasserstein one is to allow for more aggressive costs that penalize the tails of the distributions. To support this claim, the authors point at theoretical applications e.g. "theoretical guarantees for parameter estimation" and "convergence of hypoercive differential equations".

Answer

We would like to clarify one point regarding the motivation of our paper. In the previous version, we explicitly stated that

In particular, as illustrated by Equation (5), different growth regimes of ϕ\phi induce different sensitivities to discrepancies carried by a small fraction of mass.

Therefore, the mathematical motivation for this flexibility is directly illustrated by Equation (5), which shows how the choice of ϕ\phi affects the sensitivity to a small amount of transported mass.

Then, we continue state

This flexibility has already led to improved theoretical guarantees....

Therefore, the two works by R1 and R2 were cited to provide examples of previous theoretical applications of Orlicz-Wasserstein distances, rather than to justify the mathematical statement above. In particular, the use of Orlicz-Wasserstein distances is explicitly mentioned in the abstract of R1.

...For this, we develop new differential-privacy-inspired techniques based on Rényi divergences with Orlicz--Wasserstein shifts, which allow us to sidestep longstanding challenges for proving fast convergence of hypocoercive differential equations

Reference:

R1: Altschuler and Chewi (2024): Faster high-accuracy log-concave sampling via algorithmic warm starts

R2: On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances

W2:

While this can be a theoretical benefit, it comes with an additional computational cost. In my opinion this point is overlooked in the introduction as the authors hide the need to solve the minimization problem that each Orlicz-Wasserstein (OW) distance entails in the landau term of the complexity. In other words, it should be made clear that unless the phi function has a specific form, the sliced Orlicz-Wasserstein (SOW) is always more expansive than a Sliced Wasserstein (SW). Moreover, the cases presented in the paper that are easy to handle are the ones associated to polynomial functions, which are connected to standard Wasserstein distances. This aspect also feels overlooked in the paper.

Answer

We acknowledge that our use of the term "quasi-linear" in the Introduction was not sufficiently clear. We have therefore revised this part of the Introduction to make the computational complexity more explicit. We thank the reviewer again for this suggestion!

However, we respectfully disagree with the reviewer's comment that we overlooked the computational cost of SOW and only considered cases that are easy to handle. We would like to clarify the following points.

  1. In Section 3.5, we already provide a clear discussion of the computational aspects of SOW. More specifically, at the beginning of this section, we wrote:

We approximate the spherical integral by Monte Carlo sampling and compute the projected OW distances by bisection when no closed-form expression is available.

Moreover, we explicitly include the number of bisection iterations TT in our computational complexity analysis. It is worth noting that the error is reduced exponentially with TT. In practice, we observe that we just need at most 30−4030-40 iterations for bisection in general to achieve machine precision error. Appendix G.1 also provides the detailed computation for a case where bisection is not needed.

  1. As we also explained in our response to reviewer 9EGC, one of the main reasons for choosing the two Orlicz functions in our experiments was to consider two different cases: one where bisection is required and another where it is not. In fact, in Section 4.1, where we compare the computational speed of SOW and OW, we explicitly wrote:

Here, we consider ϕ1(x)=x2+x42\phi_1(x)=\frac{x^2+x^4}{2}, and ϕ2(x)=ex2−1e−1\phi_2(x)=\frac{e^{x^2}-1}{e-1} as two Orlicz functions. ϕ1\phi_1 additionally admits an exact closed-form per-projection solve for SOW (no bisection at all).

Therefore, the need for bisection and its additional computational cost were already taken into account in our experiments.

On the other hand, the final concern relates to the experimental section. Since it overlaps with the questions section, we would like to postpone our responses to this section.

AuthorsOpenReview ↗

Reponses to Reviewer's Questions (Part 1)

Q1: Aside penalizing more aggresively the tails of the distributions, is there any explicit benefit that may outweight the need to solve a minimization problem for each slice? The theoretical properties presented are closely related to the ones that the standard SW possesses, what makes the proposal new?

We first thank the reviewer for this very insightful question! In our work, we propose SOW as a generalization of SW. The properties established for general SOW also hold for SW under the corresponding assumptions. Therefore, it is natural that some of their theoretical properties are closely related. Hence, from a theoretical point of view, an interesting question is: under this generalization, how do the properties of SW change? For example, in our work, we have shown that the minimax convergence rates for estimating SOW under the compact support assumption depend on the choice of the Orlicz function ϕ\phi.

Moreover, in our study beyond compact support (Appendix H), we observe another interesting difference. For classical SW, suitable moment conditions can ensure empirical convergence (R2, theorem 1). However, for SOW, we provide a counterexample showing that, for some choices of ϕ\phi, even finite moments of all polynomial orders are not sufficient to guarantee convergence. In general, we believe that understanding how the choice of ϕ\phi affects the theoretical properties is itself an interesting research question.

From a practical point of view, we are particularly interested in two-sample testing. Among the existing approaches, MMD has been widely studied, but its performance can depend on the choice of kernel and its parameters. Recently, R1 studied two-sample testing using SW, showing that it can achieve good performance without kernel tuning while preserving the geometric structure of optimal transport. However, we believe that the possibility of choosing a suitable function can sometimes be useful. In fact, as shown in Figure 2 of R1, increasing the number of projections improves testing power, but the improvement becomes smaller when the number of projections is already large.

Inspired by this observation, we ask whether we can improve testing power by changing the distance itself, rather than only increasing the number of projections. This is one of our motivations for considering the larger family of SOW distances. Our experimental results suggest that the flexibility provided by the choice of ϕ\phi can be useful in certain situations. More specifically, in our new experiments, we use the Huber function (see its definition in our response to Q1 from reviewer 9EGC), which is piecewise quadratic and linear. This choice allows us to control transport costs over different ranges of quantile gaps.

Of course, we are still far from fully understanding how the choice of ϕ\phi affects testing power. We believe this opens an interesting direction for future research, similar to the question of how the choice of kernel affects the performance of MMD-based tests.

Finally, SOW is different from sliced convex transport costs (see our response to Q3). Besides, unlike SOW, sliced convex transport costs do not necessarily define a metric. Therefore, they cannot be considered as a general family of metrics extending SW

Reference:

R1: Tran et Schreuder (AISTATS 2026): Minimax-Optimal Two-Sample Test with Sliced Wasserstein

R2: Fournier and Guillin (2014) On the rate of convergence in Wasserstein distance of the empirical measure

AuthorsOpenReview ↗

Responses to Reviewer's Questions (Part 2)

Q2 Why the experimental results consider only the W2W_2 distance and not a W6W_6 distance, the W∞W_\infty distance or a distance induced by $h(|x-y|) $with h convex, increasing and null at zero? These metrics have a closed formula solution for one dimensional problems, see Santambrogio, F. (2015). Optimal Transport for Applied Mathematicians: Calculus of Variations.

Answer: We thank the reviewer for this very interesting suggestion, especially the suggestion to consider transport costs induced by a convex function h(∣x−y∣)h(|x-y|), as these may be confused with Orlicz-Wasserstein distances, although the two constructions are different.

Taking into account the reviewer's suggestion, we have added several baselines to our previous experiments. More specifically, we now compare SOW with SW6\mathrm{SW}_6 and SW∞\mathrm{SW}_\infty, in addition to SW2\mathrm{SW}_2. We also consider sliced convex transport costs, denoted by SC(ϕ1)\mathrm{SC}(\phi_1) and SC(ϕ2)\mathrm{SC}(\phi_2), where the transport costs are induced by the same Orlicz functions used for SOW. Moreover, thanks to the feedback from Reviewer 9EGC, we extended the number of replications from 200200 to 10001000 and also reported the error bars.

The results in Figure 4 show that, with a suitable choice of Orlicz function, SOW can still achieve higher empirical power than the considered baselines, including SW with larger values of pp.

On the other hand, to further demonstrate that the flexible choice of ϕ\phi can lead to better practical results, we have designed two new experiments: a synthetic two-sample testing experiment, where we also explain the intuition behind the choice of the Orlicz function, and a K-nearest neighbor classification experiment for distributions. We invite the reviewer to see the revised experimental section for more details.

Our goal is to demonstrate that the flexibility in choosing ϕ\phi can improve performance in certain situations. We do not claim that SOW always outperforms SW since SW is itself a special case of SOW.

We thank the reviewer again for their suggestions, which has helped us improve our experimental comparisons!

Q3: How is the SOW or OW induced by (x2+x4)/2(x^2+x^4)/2 connected to W2W_2, W4W_4 or to the Wasserstein distance induced by the cost (x−y)2+(x−y)4(x-y)^2+(x-y)^4?

Answer:

Regarding the first part, the connection with W2\mathrm{W}_2 and W4\mathrm{W}_4, we have already provided the detailed computation in Appendix G.1. Briefly, in the one-dimensional setting, we set A=W22A=\mathrm{W}_{2}^{2} and B=W44B=\mathrm{W}_{4}^{4}. Consequently, Wϕ=A+A2+8B4\mathrm{W}_{\phi}=\sqrt{\frac{A+\sqrt{A^2+8B}}{4}}.

Regarding the second part, the connection with the optimal transport cost induced by (x−y)2+(x−y)4(x-y)^2+(x-y)^4, let us denote this cost by CϕC_\phi.

Clearly, in the one-dimensional setting,

$C_\phi=\inf_{\pi\in\Pi(\mu,\nu)} \int\left(|x-y|^2+|x-y|^4\right)d\pi(x,y)=A+B.$

Consequently, we can see that, in general, Wϕ\mathrm{W}_{\phi} is completely different from CϕC_\phi.

Q4: Why in the second picture of figure 1 there is no SW in the comparison?

Answer:

The reason comes from the observation that, when we choose an Orlicz function ϕ\phi for which a closed-form expression exists, the bisection process is not necessary. Therefore, the computation time can be very close to that of SW⁡\operatorname{SW}. For this reason, we originally compared only SOW⁡ϕ1\operatorname{SOW}_{\phi_1} (without bisection) and SOW⁡ϕ2\operatorname{SOW}_{\phi_2} (with bisection).

However, to avoid any misunderstanding, we have added SW⁡\operatorname{SW} to the second panel of Figure 1. The results are consistent with our explanation above. Besides, we would like to take this opportunity to thank the reviewer for the minor comment regarding the legend in the first panel of Figure 1. We have fixed it in the revised version!

We hope that our responses and modifications have addressed the reviewer's concerns. We would be happy to discuss these points further and answer any additional questions.

Reviewer_9EGCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This submission proposes a slicing procedure for the Orlicz-Wasserstein distance and studies the topological and statistical properties of this sliced distance. In addition, an algorithm is proposed to compute the sliced distance which leverages a simplied formula in the case where the marginals are one-dimensional.

Explicitly, given a convex and strictly increasing function ϕ:[0,+∞)→[0,+∞)\phi:[0,+\infty)\to [0,+\infty) for which ϕ(0)=0\phi(0)=0 (called an Orlicz function), the Orlic-Wasserstein distance is given by \mathrm{W}_{\phi}(\mu,\nu)=\inf_{\pi\in\Pi(\mu,\nu)} \inf \left\\{ \lambda>0 :\int_{\mathbb R^d\times \mathbb R^d}\phi(\\|x-y\\|/\lambda)d\pi(x,y)\leq 1 \right\\} and is known to serve as a metric on the the space \mathcal P_{\phi}(\mathbb R^d)=\left\\{ P\in\mathcal P(\mathbb R^d):\int_{\mathbb R^d}\phi(\\|\cdot\\|/\lambda)dP<\infty\text{ for some }\lambda>0\right\\}. With these definitions in hand, it is proposed to consider a sliced version of WϕW_{\phi} given by SOWϕ,pp(μ,ν)=∫Sd−1Wϕp(Π♯θμ,Π♯θν)dσ(θ),\mathrm{SOW}_{\phi,p}^p(\mu,\nu) = \int_{\mathbb S^{d-1}} \mathrm{W}_{\phi}^p(\Pi_{\sharp}^{\theta}\mu,\Pi_{\sharp}^{\theta}\nu)d\sigma(\theta), where Π♯θ\Pi_{\sharp}^{\theta} denotes the pushforward of the measure through the map ⟨⋅,θ⟩\langle \cdot, \theta\rangle and σ\sigma is the surface measure on the d−1d-1-sphere, Sd−1\mathbb S^{d-1}. The notation convention here is to emphasize that if ϕ(t)=tp\phi(t)=t^p with p≥1p\geq 1 one recovers the sliced pp-Wasserstein distance.

With this it is shown that SOWϕ,p\mathrm{SOW}_{\phi,p} defines a metric on Pϕ(Rd)\mathcal P_{\phi}(\mathbb R^d) and that SOWϕ,p(μn,μ)→0\mathrm{SOW}_{\phi,p}(\mu_n,\mu)\to 0 if and only if sup⁡θ∈Sd−1Wϕ(Π♯θμn,Π♯θμ)→0\sup_{\theta \in \mathbb S^{d-1}}\mathrm{W}_{\phi}(\Pi_{\sharp}^{\theta}\mu_n,\Pi_{\sharp}^{\theta}\mu)\to 0. In particular, SOWϕ,p→0\mathrm{SOW}_{\phi,p}\to 0 implies that μn\mu_n converges weakly to μ\mu and the reverse implication holds if μ\mu and the sequence are supported on the same compact set. Moreover, the following comparison formulas hold: SOWϕ,p(μ,ν)≤Wϕ(μ,ν)\mathrm{SOW}_{\phi,p}(\mu,\nu)\leq \mathrm{W}_{\phi}(\mu,\nu) and, if ϕ1,ϕ2\phi_1,\phi_2 are Orlicz functions satisfying ϕ1≤ϕ2\phi_1\leq \phi_2 pointwise, SOWϕ1,p(μ,ν)≤SOWϕ2,p(μ,ν)\mathrm{SOW}_{\phi_1,p}(\mu,\nu)\leq \mathrm{SOW}_{\phi_2,p}(\mu,\nu).

Next, the properties of the plug-in estimator of SOWϕ,p(μ,ν)\mathrm{SOW}_{\phi,p}(\mu,\nu), SOWϕ,p(μ^n,ν^m)\mathrm{SOW}_{\phi,p}(\hat \mu_n,\hat \nu_m) where ν^m=1n∑i=1nδXi\hat \nu_m=\frac 1n\sum_{i=1}^n\delta_{X_i} where X1,…,XnX_1,\dots,X_n are i.i.d. samples from μ\mu and similarly for ν^m\hat \nu_m, are analyzed under the assumption that μ\mu and ν\nu are supported in a compact set with diameter DD. Letting ωϕ(u)=1ϕ−1(1/u)\omega_{\phi}(u)=\frac{1}{\phi^{-1}(1/u)} for u∈(0,1]u\in(0,1] with the convention that ωϕ(0)=0\omega_{\phi}(0)=0 it is shown that E[∣SOWϕ,p(μ^n,ν^m)−SOWϕ,p(μ,ν)∣]≤CpD(ωϕ(n−1/2)+ωϕ(m−1/2))\mathbb E\left[|\mathrm{SOW}_{\phi,p}(\hat \mu_n,\hat \nu_m) -\mathrm{SOW}_{\phi,p}(\mu,\nu)|\right]\leq C_p D\left( \omega_{\phi}(n^{-1/2})+\omega_{\phi}(m^{-1/2})\right) and a similar result is shown to hold for the pp-th power of the sliced distance. In addition, all of these rates are shown to be minimax optimal.

Subsequently, an Algorithm for solving SOWϕ,p(μ^n,ν^m)\mathrm{SOW}_{\phi,p}(\hat \mu_n,\hat \nu_m) by leveraging the fact that if PP and QQ are univariate distributions with quantile functions FP−1F_{P}^{-1} and FQ−1F_{Q}^{-1},
\mathrm{W}_{\phi}(P,Q)= \inf\left\\{\lambda >0 : \int_0^1 \phi\left( |F_{P}^{-1}(u)-F_{Q}^{-1}(u)|/\lambda \right)du\leq 1\right\\}. It follows that SOWϕ,p(μ^n,ν^m)\mathrm{SOW}_{\phi,p}(\hat \mu_n,\hat \nu_m) can be solved by taking a Monte Carlo estimate of the integral using LL samples and then performing TT bisection steps to approximate the infimum over λ\lambda. The resulting output is shown to approximate SOWϕ,p(μ^n,ν^m)\mathrm{SOW}_{\phi,p}(\hat \mu_n,\hat \nu_m) in expectation with error C1/2T+1Lvarθ∼σ[Wϕp(Π♯θμ^n,Π♯θν^m)]1/2C_1/2^T+\frac{1}{\sqrt L} \mathrm{var}_{\theta \sim \sigma}[\mathrm{W}_{\phi}^p(\Pi_{\sharp}^{\theta}\hat \mu_n,\Pi_{\sharp}^{\theta}\hat \nu_m)]^{1/2} and takes O(L(dn+nlog⁡n+Tn))O(L(dn+n\log n +Tn)) time and O(L(d+n))O(L(d+n)) memory to compute if m=nm=n.

The paper concludes with a number of experiments. First, the effects of varying the Orlicz function, the sample size, the number of projections, and the number of bisection iterations on the run time and estimation accuracy is explored on synthetic data. Next, the sliced Orlicz-Wasserstein distance is compared to the sliced Wasserstein distance and MMD with two kernels for the problem of hypothesis testing. It is shown that the sliced Orlicz-Wasserstein distance can obtain higher empirical power (estimated using permutation tests) in one synthetic example and examples with MNIST and CIFAR-10 and CIFAR-10.1. A final experiment evaluates generative models using sliced Wasserstein and sliced Orlicz-Wasserstein.

Critical strengths

The paper develops a comprehensive account of the topological and statistical properties for the Sliced Orlicz-Wasserstein distance which are supported by proofs that appear correct to me. The addition of minimax optimality goes beyond "standard" statistical results one would expect in such a paper. The approximation guarantee for Algorithm 1 is also a good theoretical addition. The counterexample from Remark 1 is also surprising.

Critical weaknesses

In my opinion, the technical contributions of the paper are not clearly stated. For instance, Theorem 1 and Proposition 2 follow essentially verbatim from known results, see Proposition 1 in [2], keeping in mind that Orliz-Wassertein is a metric (cf. Lemma 3.3 in [1]), and Lemma 3.4 in [1]. Similarly, Corollary 1 and Proposition 4 follow directly from [2] once Lemma 7 and 8 have been established, as they imply that convergence under Wϕ\mathrm{W}_{\phi} implies convergence under 11-Wasserstein distance (hence weak convergence) and that the converse holds when all measures are supported in a compact set so that Theorem 1 in [2] yields both of those results (once it is shown that the Orlicz-Wasserstein distance between distributions with compact support admits a uniform bound). The discussion surrounding those results in the main text only states that Theorem 1 and Propositions 1 and 2 build on the arguments in [1]. Given this, I view the main theoretical contributions of the work as Proposition 1 and 3 (whose implications should be more clearly discussed), Theorems 2,3, and 4, Corollary 2, and Proposition 5 which is sufficient in my opinion. Additionally, in the contributions, it is stated that the paper takes the first step beyond compact support, but this is not discussed at all in the main text (except for one sentence in the conclusion).

Some of the theoretical results lack proper motivation/discussion. For instance, what are the practical implications of Propositions 1 and 2? Similarly, Proposition 3 is only used to justify that convergence under SOWϕ,p\mathrm{SOW}_{\phi,p} implies weak convergence, but this appears to be a stronger notion of convergence. On the other hand, the lead-up to Theorem 3 states that the analysis requires controlling the variation of the powered Luxemburg norm, but it is not explained why this should be case. I would recommend simplifying the discussion if possible. In particular, the sentence "For the powered-functional estimation problem, we need to control how the powered Luxemburg norm changes when a quantile-gap profile is perturbed in L1L^1" does not add much to the discussion since the reader has no context for what a quantile-gap profile is.

Finally, I am not quite convinced on the framing of the paper. In the abstract it is stated that "Our experimental results reveal the superior computational efficiency of sliced Orlicz-Wasserstein compared with Orlicz-Wasserstein. Also, in the experiments, we demonstrate the favorable flexibility of SOW distance over SW in detecting differences between distributions by comparing their performance in two-sample tests and evaluating generative models on image datasets" whereas the conclusion states that "[the experiments] provide initial evidence that suitable choices of ϕ\phi can increase sensitivity to discrepancies involving small amounts of mass transported over large distances [compared with sliced Wasserstein]". In my opinion, the second statement is much more accurate for the level of evidence provided by the small number of experiments. Moreover, it seems that by increasing the degree pp of the sliced Wasserstein distance beyond 22, one may be able to recover similar results as with the sliced Orlicz-Wasserstein framework (if the advantage can really be explained by penalizing transport over larger distances). Consequently, the advantages of the sliced Orlicz-Wasserstein framework over the sliced Wasserstein framework are not sufficiently well established in my opinion; I believe that a concrete application where sliced Orlicz-Wasserstein outperforms the sliced Wasserstein (with different choices of pp) in a moderate number of examples would be necessary to support the claims from the abstract and hence justify the theoretical development of sliced Orlicz-Wasserstein distances.

References:

  1. A. Guha, N. Ho, and X. Nguyen. On excess mass behavior in Gaussian mixture models with Orlicz-Wasserstein distances. In International Conference on Machine Learning (pp. 11847-11870). PMLR, 2023.
  2. K. Nadjahi, A. Durmus, L. Chizat, S. Kolouri, S. Shahrampour, and U. Simsekli. Statistical and topological properties of sliced probability divergences. Advances in Neural Information Processing Systems, 33, pp.20802-20812, 2020.
Questions for authors
  1. The rates derived in Section 3.3 are interesting, but, except for the case ϕ(t)=tp\phi(t)=t^p with p≥1p\geq 1, no other examples are provided for what the results look like for different choices of Orlicz functions. Can the authors provide a table of some Orlicz functions and their corresponding rates?

  2. Related to the above question, can the theoretical empirical convergence rates (or some modification thereof to account for a finite number of slices) be validated experimentally? In addition, can the theoretical scalings be added to Figure 1 so that the reader can verify that they are respected in that experiments?

  3. It is not so clear to me why ϕ1\phi_1 and ϕ2\phi_2 are used in the experiments beyond the fact that computation under ϕ1\phi_1 does not require bisection, as there is no discussion on the motivation for these choices. Can the authors clearly explain the rationale for choosing these Orlicz functions/what these choices promote in the Orlicz-Wasserstein framework?

  4. In addition to Figure 3, can the empirical level of the test also be checked to verify if the claimed level of α=0.05\alpha=0.05 is respected? It also seems to me that 200200 replications is rather small, the authors may consider a larger scale experiment. Finally, it would appear reasonable to perform multiple repetitions of the experiment and report the mean with error bars.

9 comments
AuthorsOpenReview ↗

Responses to the Reviewer's Main Concerns (1/2)

First, we would like to thank the reviewer for their time and constructive feedback. We would like to extend our discussion as follows:

W1: The motivation of the theoretical results and the presentation of the technical contribution

Answer:

We introduce the Sliced Orlicz-Wasserstein (SOW) distance as a generalization of the classical Sliced Wasserstein (SW) distance. Our main goal is to understand how the mathematical and statistical properties change under this generalization.

We start by studying whether the basic metric and topological properties are still preserved. These results can be obtained from existing results on Orlicz-Wasserstein distances and sliced probability divergences (see R1,R2). To be more specific, we try to make a link among SOW, SW, Orlicz Wasserstein (OW), and Wasserstein (see our Lemma 5 and 6 in Appendix C) so that we can leverage on the existing results. We do not consider them as our main technical contributions. Instead, we include them to provide a mathematical foundation for SOW.

We are more interested in the estimation problem. In practice, we usually do not know the underlying distributions and only have access to samples from them. Therefore, we need to estimate SOW from these samples. One natural choice is the plug-in estimator, which we study in Section 3.3. However, after obtaining the convergence rates of the plug-in estimators, a natural question is whether these rates are optimal. In other words, can we find another estimator with a better convergence rate in the worst case? This question motivates our study of minimax optimality in Section 3.4. To the best of our knowledge, these types of results have not been established before for general OW distances. Therefore, Sections 3.3 and 3.4 contain our main technical contributions. More specifically, in Appendix I, we also explain some of the difficulties that arise when extending the statistical analysis from SW to SOW.

Finally, regarding the extension beyond compact support, we discuss this problem in Appendix H. In fact, this problem is more challenging for general Orlicz functions. As shown by our counterexample in Appendix H.1, for some choices of ϕ\phi, even assuming finite moments of every polynomial order is not sufficient to guarantee empirical SOW convergence. We also put some efforts to go further in Appendix H.2. However, it is still unsolved. Since our main focus is on statistical estimation under compact support, and given the limited space in the main text, we have kept the detailed discussion in Appendix H.

Based on the reviewer's suggestions, we have rewritten the contributions in the Introduction and shortened the section on metric and topological properties (as well as the corresponding proofs in Appendix C).

Reference:

R1: Aritra Guha et al(ICML 2023): On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances

R2: Kimia Nadjahi et at (Neurips 2020): Statistical and Topological Properties of Sliced Probability Divergences

AuthorsOpenReview ↗

Responses to the Reviewer's Main Concerns (2/2)

W2: Regarding the practical advantages of SOW.

Answer:

We first thank the reviewer for their suggestion. We agree that the second statement is more accurate, and we have revised it accordingly.

However, we respectfully disagree with the comment that simply increasing the order p>2p>2 of the sliced Wasserstein distance would generally lead to similar results as the sliced Orlicz-Wasserstein framework.

Thanks to the comments from reviewer g3Xi and your suggestions, we have decided to add more baselines to our previous experiments. More specifically, for the previous two-sample testing scenarios, we have added SW⁡6\operatorname{SW}_{6} and even SW⁡∞\operatorname{SW}_{\infty}. Besides, following an interesting suggestion from reviewer g3Xi, we also compare SOW with sliced convex transport costs, which are sometimes confused with SOW, although they are different (see, for example, our reponse to Q3 of reviewer g3Xi) . The results in Figure 4 show that, with a suitable choice of Orlicz function, SOW can still achieve higher empirical power than the considered baselines.

Moreover, we have added a new synthetic setting to provide more intuition about the choice of the Orlicz function. The results in Table 1 give us evidence that increasing pp does not necessarily improve the testing power of SW⁡p\operatorname{SW}_p and (sometimes) can even lead to worse results. It is important to note that the number of projections used for SOW in this experiment is only 250, while the other sliced methods use 500 projections.

To further support this point, we have also designed a new experiment on K-nearest neighbor classification of distributions (Section 4.3). The results again show that, in some situations, simply increasing pp is not enough to obtain the same performance as SOW.

Besides, we note that unlike the power function tpt^p, an Orlicz function can have different growth rates in different ranges. In this synthetic setting, we consider the Huber function (see its definition in our answer to Q1), which is piecewise quadratic and linear. This choice allows us to control transport costs over different ranges (Table 1, Section 4.3). We believe that these experiments demonstrate the potential benefits of SOW in certain situations. They also raise an interesting direction for future research on how to choose the Orlicz function ϕ\phi according to the structure of the distributions.

Based on the reviewer's suggestions, we have added the benchmark candidates and also the new experimental setups. We would like to invite the reviewer to see the updated version (Section 4.2, Section 4.3).

AuthorsOpenReview ↗

Responses to the Reviewer's Questions

Q1: Can the authors provide a table of some Orlicz functions and their corresponding rates?

Answer

As shown in Theorems 2–4, our results provide the convergence rates for a general Orlicz function ϕ\phi. For a specific choice of ϕ\phi, the corresponding rates can be obtained from the definitions of ωϕ\omega_\phi and ψϕ,pcav\psi_{\phi,p}^{\mathrm{cav}}.

To make these results more precise, we provide the following examples. For simplicity, we consider p=2p=2 and n=mn=m.

Orlicz function ϕ(t)\phi(t)SOW⁡\operatorname{SOW}SOW⁡pp\operatorname{SOW}^p_p
t2t^2n−1/4n^{-1/4}n−1/2n^{-1/2}
t2+t42\dfrac{t^2+t^4}{2}1+1+8n4n\sqrt{\dfrac{1+\sqrt{1+8\sqrt n}}{4\sqrt n}}1+1+8n4n\dfrac{1+\sqrt{1+8\sqrt n}}{4\sqrt n}
et2−1e−1\dfrac{e^{t^2}-1}{e-1}1log⁡(1+(e−1)n)\dfrac{1}{\sqrt{\log(1+(e-1)\sqrt n)}}1log⁡(1+(e−1)n)\dfrac{1}{\log(1+(e-1)\sqrt n)}
ϕH2(t)\phi_{H_2}(t) (Huber)4n+4\dfrac{4}{\sqrt n+4}, n≥16n\ge161n\dfrac{1}{\sqrt n}

Here, the normalized Huber function with δ=2\delta=2 is defined as

\end{cases}$$ **Q2:** Can the theoretical empirical convergence rates (or some modification thereof to account for a finite number of slices) be validated experimentally? ***Answer*** To the best of our knowledge, we believe that minimax convergence rates cannot be precisely validated through numerical experiments. Indeed, "minimax" means that we consider the "best" estimator in the "worst-case" scenario over a class of distributions. More precisely, we take the infimum over all possible estimators and the supremum over all distributions in the considered class. Such a setting is difficult to reproduce in numerical experiments, since we can only test a limited number of estimators and distributions. This is also one of the reasons why establishing minimax optimality is an interesting and challenging problem in statistics. Moreover, as a theoretical sanity check, by taking $\phi(t)=t^p$, we recover the known convergence rates of the classical sliced Wasserstein distance, as discussed in Remark 2. **Q3** Can the authors clearly explain the choice of $\phi_1$ and $\phi_2$. ***Answer*** Our choice of $\phi_1$ and $\phi_2$ is mainly motivated by two reasons. First, from a *computational point of view*, we want to consider two different cases: one where bisection is not needed and another where bisection is used. More precisely, for $\phi_1(t)=(t^2+t^4)/2$, the one-dimensional Orlicz-Wasserstein distance admits a closed-form expression (see Appendix G.1). On the other hand, for $\phi_2(t)=(e^{t^2}-1)/(e-1)$, we use bisection to compute the projected distances. This allows us to show that bisection is not always necessary and to compare the computational costs of these two cases in Figure 1. Second, we want to investigate Orlicz functions with different growth behaviors. While $\phi_1$ has polynomial growth, $\phi_2$ has exponential growth. As illustrated by Equation 5, these choices lead to different sensitivities to differences involving a small amount of probability mass.Besides, $\phi_2$ is also the normalized version of Orlicz function considered in R3, where the authors show the theoretical benefit of Orlicz Wasserstein. We want to note that our goal is not to claim that one function is always better than the other, but to illustrate the flexibility of SOW. Regarding this question, we have the new experimental setup for two-sample testing, where we try our best to explain the intuition for the choice of $\phi$ (Figure 3, Table 1). Reference: R3: Altschuler and Chewi (2024): Faster high-accuracy log-concave sampling via algorithmic warm starts **Q4:** Can the empirical level of the test also be checked to verify if the claimed level of is respected? It also seems to me that 200 reps is rather small, the authors may consider a larger scale experiment. Finally, it would appear reasonable to perform multiple repetitions of the experiment and report the mean with error bars. ***Answer*** Regarding the empirical level (or Type I error), we would like to point out that our experiments use permutation tests, which are known to control the Type I error under the null hypothesis. More precisely, under $H_0:\mu=\nu$, the pooled observations are exchangeable with respect to permutations of the sample labels. This is a finite-sample guarantee, which also holds when using a finite number of random permutations with the appropriate procedure (see R4 proposition 1). Regarding the number of replications, we thank the reviewer for this suggestion. We have increased the number of independent replications from 200 to 1000 in our two-sample testing experiments. We also report the empirical power together with its Monte Carlo standard error as shown in Table 1, Figure 4 of the revised manuscript. Reference: R4: Antonin Schrab et al (JMLR 2023): MMD Aggregated Two-Sample Test
Reviewer_j38MOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a sliced version of the Orlicz–Wasserstein metric and investigates its theoretical and computational properties. The motivation is clear, and the paper is generally well written and easy to follow.

Critical strengths

The overall structure closely follows that of Nadjahi et al. (2020), establishing the topological, statistical, and computational properties of the proposed distance. The theoretical analysis is thorough and covers essentially all the important aspects of the proposed metric. The results are clearly stated and presented in a systematic manner.

The theoretical contributions are also supported by numerical experiments, which provide a welcome complement to the theoretical results.

Overall, I have a positive assessment of the paper. The theoretical development is comprehensive, the presentation is clear, and the experimental results further strengthen the contribution. I believe the paper is suitable for acceptance in its current form.

Critical weaknesses

None.

Questions for authors

None.

1 comment
AuthorsOpenReview ↗

Responses to Reviewer's Feedback

We sincerely thank the reviewer for their time and feedback. We appreciate the positive assessment of the paper

Comments on the paper

1 comment

Dear AC and Reviewers,

We thank the reviewers for their time, their constructive comments, and their participation in the discussion.

As we approach the end of this phase, we would like to summarize the main revisions and additional results provided during the discussion.

1. Revisions following the initial reviews

a) Presentation

Taking into account the reviewers’ comments, we have revised the abstract and the contribution section to clarify the scope and main contributions of our paper.

Reviewer g3Xi’s comment about the legend in the first panel of Figure 1 has also been addressed.

b) Experiments

Following Reviewer g3Xi’s suggestion, we have added SW to the second panel of Figure 1.

We have also expanded the experimental comparisons to include SW6SW_6, SW∞SW_\infty, and sliced convex transport costs induced by ϕ1\phi_1 and ϕ2\phi_2. Figure 4 shows cases where SOW with the considered choices of ϕ\phi achieves higher empirical power than the tested SW baselines in both synthetitc and real data settings.

We added a synthetic experiment to illustrate the intuition behind choosing the Huber function. In this setting, Table 1 reports higher empirical power for SOW with the Huber function than for the tested SWpSW_p baselines, with p∈{1,1.25,1.5,1.75,2,3,4,5,6,7,8,9,10,∞}p\in\{1,1.25,1.5,1.75,2,3,4,5,6,7,8,9,10,\infty\}, and the sliced convex transport cost induced by the same Huber function. SOW uses 250 projections, while the other sliced methods use 500.

We also added a synthetic K-nearest neighbor classification experiment for distributions. Table 2 reports higher test accuracy for SOW with the Huber function than for the considered SW baselines.

Besides, we increased the number of replications to 1000 and reported uncertainty estimates thanks to reviewer 9EGC's suggestion.

These experiments provide evidence that the flexibility in choosing ϕ\phi can be useful in the considered settings. Although we are still far from providing specific and precise guidance on choosing ϕ\phi, we believe that this is an interesting and challenging direction for long-term research.

2. Follow-up points raised during the discussion

Reviewer 9EGC suggested adding a short explanation of the technical difficulty after the results in Section 3.3, with a reference to Appendix I. They also requested further explanations of the choices of ϕ1\phi_1 and ϕ2\phi_2, and a revision of the statement at line 530 on page 10.

We have provided proposed text in our responses. Since changes around the wrapfigure environment affect the page layout, we will incorporate these clarifications into the final revision.

Following Reviewer 9EGC’s suggestion, we also provided an expected error bound combining the statistical, slicing, and bisection errors for the computed quantity. We have provided in our response and we will include this bound in the final revision.

Besides, we have reported empirical Type I error estimates for the Synthetic, MNIST, and CIFAR-10 settings in the discussion, using 1000 replications and 199 permutations at the nominal level of 0.05. We will include these results in the final revision.

Reviewer g3Xi also suggested acknowledging references on the computation of W∞W_\infty and adding Santambrogio’s book as a general OT reference. We will include these references in the revised manuscript.

For the moment, reviewer 9EGC indicated that they had no further questions at the moment and would reassess their initial evaluation, noting that most of the requested clarifications had been added. Reviewer g3Xi stated that the response had convinced them and that they were leaning toward a more positive evaluation which will be granted after the double check.

We thank all reviewers again for their time and the helpful discussions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.