Exact and Inexact Marginal Penalties: When Does the Wasserstein Autoencoder Recover the Wasserstein Distance?

Primal optimal transport (OT) minimizes a transport cost over couplings with prescribed marginals. Parameterizing a coupling by a conditional distribution enforces one marginal by construction and leaves the other as an explicit constraint. The Wasserstein autoencoder relaxes this remaining constraint into a divergence penalty. We characterize when the resulting penalized objective recovers the OT cost at a finite coefficient $\lambda$. On bounded domains and under an active constraint, $f$-divergences with $f$ differentiable at $1$, as well as squared maximum mean discrepancy, strictly underestimate the cost for every finite coefficient. For $f$ satisfying a generalized Pinsker inequality, the optimal-value gap is $\Theta (1/\lambda )$ as $\lambda \to \infty$. In contrast, the square roots of such $f$-divergences and the 1-Wasserstein distance recover the OT value and global minimizers above explicit finite thresholds, and for square roots of the Kullback-Leibler and chi-square divergences exactness extends to unbounded priors. The dichotomy follows from comparing reconstruction descent with penalty growth in the Wasserstein space of joint distributions. Uniform exactness over a decoder class further yields statistical guarantees for learning the pushforward model. Simulations with computable ground truth illustrate the dichotomy, while CelebA experiments compare latent matching and local image redundancy across penalty classes.