Wasserstein Singular Vectors and Metric Dowker Duality for Interpretable Feature Geometry

The Wasserstein singular vector (WSV) problem asks for a metric on the samples and a metric on the features of a data matrix such that each reproduces the other through optimal transport. We prove existence for the general, unregularized WSV problem, for almost every matrix, and match degenerate solutions to lumpings of the matrix. While they determine each other, the WSV sample and feature metrics do not generally agree. We show that their Gromov–Hausdorff and Gromov–Wasserstein distance is bounded by the spread of the features over the samples. When this spread is small, every feature has a location among the samples and the features together form a metric space that mirrors the sample geometry. This is a global and label-free notion of an interpretable feature geometry. All our mathematical results are machine-verified in Lean 4. Spread is also a quantity a model can be trained on. We regularize WSV spread in autoencoders on geometric toy data and on Gemma-2-2b residual-stream activations along a prompt manifold. As the theory predicts, this allows us to recover the sample geometry on the feature side. We show that interpretable feature geometry induced by $L^1$ regularization can also be explained by low WSV spread.