- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper deals with the theoretical question whether the OT solution of an encoder arising in Wasserstein autoencoders is consistent with a regularized version of the problem. In the regularized version, one minimizes a transport cost but relaxes the marginal constraint to a soft penalty. The paper studies for which "discrepancies" such regularized version admits the same optimal encoder as the original OT problem. Somewhat surprisingly, it does not work for f-divergences but for its square root. The paper validates its theory on analytic low dimensional example and CelebA.
Critical strengths
-
The proofs and arguments seem correct as far as I checked.
-
The placement wrt prior work is rigorous, table S7 is good to read.
-
It is interesting to see that this argument works for the square root of such f divergences.
Critical weaknesses
-
I do not think the Wasserstein autoencoder formulation is SOTA or used much anymore. Most of the literature now uses conditional normalizing flows/diffusion models for which this theory is largely irrelevant since they fix the marginals automatically. The case of generic autoencoders is not covered since the paper explicitly studies the OT formulation. Generally, it is not known whether minimizing a Wasserstein inspired loss term (either via the dual or primal formulation) really leads to recovery of the actual transports for a learned NN. Most strongly, my recommendation would change if the authors could argue that it is relevant to more papers than I currently believe. I currently do not think it is impactful/significant enough to be published at ICLR.
-
I find the use of GANs to approximate the square-root of the f-divergence needs some justification, in particular since for instance Wasserstein GANs do not approximate the Wasserstein distance [1]. It seems to be done even in the low dimensional examples and therefore one could also resort to binning?
-
The writing is very hard to follow, many sentences are hard to parse for me personally and made it quite hard to read. Examples include: " The proofs proceed by perturbing a feasible,nearly optimal joint distribution ... mixture segment,where .. is the joint distribution induced by the quantile encoder that restricts the prior to the θ-fraction (θ∈(0,1)) of latent codes minimizing the reconstruction cost c(x,g(z)),yielding a latent marginal π1♯ρ with density at most 1/θ with respect to PZ "
"Fixed-decoder simulations compare reconstruction estimates with computable OT ground truth and illustrate both sides of the finite-coefficient dichotomy. "
- Some of the mathematical proofs are standard, e.g., Proposition 3.9 but are not cited or deemed as such. I would appreciate if it was clearer whether standard results were marked as such (see [2])
[1] Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance), Stanczuk et al, 2021
[2] On Choosing and Bounding Probability Metrics, Gibbs and Su, 2002.
Questions for authors
see weaknesses
Thank you for the detailed responses and clarifications. I appreciate the additional experiments, particularly the extended ablations on penalty coefficients, which help clarify the practical differences between the proposed penalties. The explanations addressing my specific questions were also helpful. That said, my main concerns remain. While the theoretical results are interesting, the empirical advantages appear limited. In particular, the additional experiments suggest that both penalties achieve comparable performance when properly tuned, with the main benefit of the square-root penalties being reduced sensitivity to coefficient selection. While this is a useful property, I am still not convinced that it represents a sufficiently significant practical contribution. I therefore appreciate the authors' efforts but will maintain my original rating.