SOLA: Stochastic One-Loss Autoencoders with Population-Derived Latent Geometry

Variational autoencoders (VAEs) typically obtain stochastic latent representations by balancing reconstruction against a Kullback–Leibler (KL) regularization term, introducing a problem-dependent trade-off that can lead to posterior collapse. We ask whether a stochastic generative autoencoder can retain a principled KL interpretation without optimizing a KL loss at all. We introduce the *Stochastic One-Loss Autoencoder* (SOLA), which derives both latent stochasticity and sampling geometry directly from the encoded population. SOLA combines a rank-reduced orthogonal latent representation with random latent mixing, inducing Gaussian perturbations whose covariance is determined by the geometry of each encoded batch. We show that the resulting stochastic representation admits a geometry-matched Gaussian reference for which the batch-averaged KL divergence is exactly constant for fixed bottleneck dimension and noise scale, independently of the network parameters and learned singular spectrum. Its optimization gradient is therefore identically zero. Accordingly, SOLA is trained using reconstruction alone while retaining a stochastic latent representation. After training, the learned batch geometries are aggregated into a fixed global representation that supports deterministic inference and a population-derived Gaussian sampling rule, without a learned variance head, KL weight, or KL schedule. Across six synthetic, structured, and real-world datasets, SOLA achieves competitive reconstruction and generation using the same noise scale $\sigma =0.1$ throughout. In contrast, both the preferred VAE KL weight $\beta$ and the qualitative response to changing it vary substantially across problems. Controlled ablations separate the effects of dimensionality reduction, orthogonal latent geometry, and stochastic mixing, while empirical diagnostics assess the latent statistics underlying the Gaussian sampling approximation. Comparisons with modified VAE objectives and alternative generative autoencoders, including KL annealing, Free Bits, $\sigma$-VAE, capacity-controlled $\beta$-VAE, RAE, WAE, and VQ-VAE, further show that competitive generative performance can be obtained without explicitly optimizing a prior-matching objective or fitting a separate latent density for SOLA's native generator. The code is open-source and available in the supplementary material.