Mind the Gaps: Multiscale Disentanglement through InfoMax Auto-Encoders

A primary goal for unsupervised learning is to discover disentangled representations of latent factors. Traditionally, such disentanglement is obtained by enforcing constraints on the prior distribution over latents, but this typically entails a loss of reconstruction quality. Here, we show that such constraints are unnecessary. Leveraging an explicit connection between mutual information and MMSE estimators, we develop a novel objective for information maximization which leads to disentangled representations without reliance on prior-matching constraints. Under specific assumptions on the independence, separability, and scale separation of the generative factors, we prove that this objective, paired with a factorized Gaussian encoder, yields multiscale disentanglement. Numerical experiments on both synthetic and natural image datasets support this theoretical prediction. More generally, our approach shows that the combination of classic information-theoretic principles with the expressive power of modern machine learning offers a fruitful path for the development of new models and algorithms.