A Minimum-Variance Perspective on the Encoder and ELBO in VAEs

Variational Autoencoders (VAEs) are widely used for generative modeling, denoising, super-resolution, and compression, and are characterized by their encoder-decoder architecture. Yet, from the perspective of the generative model, the encoder is not required: the marginal likelihood is defined solely by the decoder and the latent prior, while the encoder introduces additional trainable parameters. Then, *why should an encoder be part of a generative model?* In this paper, we show that the encoder-decoder structure of VAEs can be derived from a minimum-variance principle and it is not only an inherited architectural choice. Starting from weak assumptions and considering the finite-budget Monte Carlo estimation of marginal likelihoods, we use a classical result from importance-sampling theory to derive the variance-minimizing proposal distribution. This proposal is input-dependent, and its amortized implementation necessarily induces an encoder that maps observations $x$ to a distribution over latent variables $z$, followed by a decoder that generates $x$ from $z$. Thus, the encoder-decoder structure emerges as a variance-optimal construction for likelihood estimation. We further show that the ELBO is consistent with the same principle: beyond providing a tractable lower bound on the marginal log-likelihood, its optimization is aligned with reducing the variance of the corresponding importance-sampling estimator and with approaching the variance-minimizing proposal. This establishes a common statistical foundation for both the VAE architecture and its training objective. Experiments on controlled latent-variable benchmarks confirm the predicted effect: encoder-based models achieve lower importance-sampling estimator variance and higher likelihood than encoder-free alternatives, with the advantage increasing as the latent dimensionality grows.